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Abstract. We propose a new definition of mctastability of Markov processes 
on countable state spaces. We obtain sufficient conditions for a sequence of pro- 
cesses to be metastable. In the reversible case these conditions are expressed in 
terms of the capacity and of the stationary measure of the metastable states. 



1. Introduction 

In the framework of non-equilibrium statistical mechanics, metastability is a 
relevant dynamical phenomenon taking place in the vicinities of first order phase 
transitions. There has been along the years several proposals of a rigorous math- 
ematical description of the phenomenon starting with Lebowitz and Penrose [16] 
who derived the canonical free energy for Kac potentials in the Van der Waals limit. 
The seminal paper of Cassandro, Galves, Olivieri and Vares [6] proposed a path- 
wise approach to metastability which highlighted the underlying Markov structure 
behind metastability which is exploited here. In the sequel, Scoppola |19j examined 
the metastable behavior of finite state space Markov chains with transition proba- 
bilities exponentially small in a parameter. More recently, Bovier and co-authors 
([5] and references therein) presented a new approach based on the spectral prop- 
erties of the generator of the process. We refer to [18| for a recent monograph on 
the subject. 

We propose in this article an alternative formulation of metastability for se- 
quences of Markov processes on countable state spaces. Informally, a process is 
said to exhibit a metastable behavior if it remains for a very long time in a state 
before undergoing a rapid transition to a stable state. After the transition, the 
process remains in the stable state for a period of time much longer than the time 
spent in the first state, called for this reason metastable. In certain cases, there are 
two or more "metastable wells" with the same depth, a situation called by physi- 
cists "competing metastable states" . In these cases, the process thermalizes in each 
well before jumping abruptly to another well where the same qualitative behavior 
is observed. 

To describe our approach, denote by Ejy, N > 1, a, sequence of countable spaces 
and by (^^v : > 1) a sequence of positive real numbers. For each > 1, consider 
a partition £]y, . . . , Ajv of En and a i?Ar-valued Markov process {?]^ ■ t >0}. 
Fix a state in 1 < x < k. We say that the sequence of Markov processes 
{Vt^ : i > 0}, TV > 1, exhibits a tunneling behavior in the time scale {9n ■ N > 
1) with metastates S.]^, . . . , £.'^, attractors and asymptotic behavior 
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described by the Markov process on 5 = {1, ... , k} with rates {r{x, y) : x,y & S} 
if the following three conditions are fulfilled: 

(1) For every 1 < x < k, starting from a state r]^ in with overwhelming 
probability, the process {rj^ ■ t > 0} reaches before attaining [Jy-^^ ^at- 

(2) Let {Xf : i > 0} be the process = 4-^(77^"), where {r/f" : t > 0} 
is the trace of the Markov process {774 : t > 0} on En = Ui<k<k ^^"^ 
where 5'Ar(?7) = J2i<x<k ^^{v £ ^n}- The speed up process {Xj^^ ■ t > 0} 
converges to the Markov process on S which jumps from x to y at rate 
r{x,y). 

(3) Starting from any point of E^r, the time spent by the speed up Markov 
process {rjte,^ : t > 0} on the set A^r in any time interval [0, s], s > 0, 
vanishes in probability. 

All the terminology used in the previous definition is explained in the next sec- 
tion. Condition (1) states that the process thermalizes in each set £^ before reach- 
ing another metastate set y ^ x. The assumption of the existence of an 
attractor can clearly be relaxed, but is satisfied in several interesting examples, 
as in the condensed zero-range processes [3l |4] which motivated the present work. 
Condition (2) describes the intervalley dynamics and reveals the loss of memory of 
the jump times from a well to another, put in evidence in [B]. In condition (3) we 
assume that the starting point belongs to En- It may therefore happen that the 
discarded set An hides wells deeper than the wells E%, 1 < x < k, but which can 
not be attained from En- When we remove in this condition the assumption that 
the starting point belongs to En, we say that the process exhibits a metastable 
behavior, instead of a tunneling behavior. In this case, the wells 1 < a; < k are 
the deepest ones. 

In contrast with the pathwise approach to metastability [5], the present one 
does not give a precise description of the saddle points between the wells nor of 
the typical path which drives the system from one well to another. Its descrip- 
tion of metastability is in some sense rougher, but keeps the main ingredients, as 
thermalization and asymptotic Markovianity. 

The main results of this article, stated in the next section, establish sufficient 
conditions for recurrent Markov processes on countable state spaces to exhibit a 
tunneling behavior. In the reversible case, these sufficient conditions can be ex- 
pressed in terms of the capacity and of the stationary probability measure of the 
metastates. 

A theory is meaningless if no interesting example is provided which fits in the 
framework presented. Besides the mean field models considered in [B] and the 
Freidlin-Wentsell Markov chains proposed in [18], which naturally enter in the 
present framework, we examine in [3l |4j a new class of processes which exhibit a 
metastable behavior. This family, known as the condensed zero-range processes, 
have been introduced in the physics literature [51 [T31 [S] to model the Bose-Einstein 
condensation phenomena. It has been proved in several different contexts [TH (TUl H] 
that, above a critical density, all but a small number of particles concentrate on 
one single site in the canonical stationary states of these processes. In [3j H] we 
prove that, in the reversible case, the condensed zero range processes exhibit a 
tunneling behavior by showing that in an appropriate time scale the condensed site 
evolves according to a random walk on S. We also prove that the jump rates of 
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the asymptotic Markov dynamics can be expressed in terms of the capacities of the 
underlying random walks performed by the particles. 

The article is organized as follows. In Section [21 we introduce the notation, the 
definitions and state the main theorems. In Section [3] we present some elementary 
examples which justify the definitions proposed. In Sections IH [5l we prove the 
main results. Finally, in Section [6l we prove some results on the trace of Markov 
processes needed in the article and which we did not find in the literature. 

2. Notation and Results 

Fix a sequence {En : N > 1) of countable state spaces. The elements of En 
are denoted by the Greek letters 77, ^. For each > 1 consider a matrix Rn ■ 
En X En K such that i?Ar(77,^) > for rj ^ -00 < RN{ri,ri) < and 
^fe-Eiv ^i^i^'O = for all 77 G En- Denote by Ln the generator which acts on 
bounded functions / : En ^ M as 

{LNfKv) = E ^^('?'^) {/(^) - /(^)} • (2.1) 

Let {77^ : t > 0} be the minimal right-continuous Markov process associated to 
the generator Ln- We refer to [71 [Til [17] for the terminology and the main facts 
on Markov processes alluded to in this article. It is well known, for instance, that 
{■q^ : i > 0} is a strong Markov process with respect to the filtration {J^^ : t > 0} 
given by jPf = a{r]^ ■- s < t)- To avoid unnecessary technical considerations, we 
assume throughout this article that there is no explosion. 

Denote by D{M.+ , En) the space of right-continuous trajectories e : M+ En 
with left limits endowed with the Skorohod topology. Let P^, 77 G En, be the 
probability measure on D{M.^, En) induced by the Markov process {7;^ : t > 
0} starting from rj. Expectation with respect to is denoted by and we 
frequently omit the index N in P^, E^. 

For every > 1 and any subset A C En, denote by ta ■ D{R+,En) K+ the 
hitting time of the set A: 

TA ■-= inf {s > Q : Es £ A] , 

with the convention that = 00 if e^. A for all s > 0. When the set A is a 
singleton {77}, we denote T{^} by t^. This convention is adopted everywhere below 
for any variable depending on a set. In addition, for each t > 0, define the additive 
functional 7^^ : Z)(M+,i?) i-^ M+ as the amount of time the process stayed in the 
set A in the interval [0, i]: 

T;-^ I l{e, G^}ds, i>0, (2.2) 

where 1{-B} stands for the indicator of the set B- 

A sequence of states r] = [rj^ G En : > 1) is said to be a point in a sequence 
A of subsets of En, A = (An En ■- N > 1), if 77^ belongs to An for every N > 1. 
For a point r/ = (7;^ € En ■- N > 1) and a set ^1 = {An En ■- N > 1), denote by 
Tr,, Ta, the hitting times of the sets {rj}. A: 

Ti rjiN rji rjiN 

ri ^ J^r, ■= T-^" I J^-A ^ -La •= ^Ajv ■ 
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For any sequence of subsets A — {Aj^ C En : N '>V),'S ~ {Fn C En : N > 1), 
denote by Ta (3^) the time spent on the set 3^ before hitting the set A: 

Ta{-J) = T^{-J) l{77f e FN}ds . 

Jo 

2.1. Valley with attractor. We introduce in this subsection the concept of valley. 
Intuitively, a subset W of the state space En is a valley for the Markov process 
: t > 0} if the process starting from W thermalizes in W before leaving W at 
an exponential random time. 

To define precisely a valley, consider two sequences W, S of subsets of En, the 
second one containing the first and being properly contained in En- 

W = (Wn ^En ■■ N >l), 'B ^ (Bn ^ En ■■ N >1) , Wn ^ Bn ^ En ■ (2.3) 

Fix a point | = {^n G Wn : iV > 1) in W, a sequence of positive numbers 
e ^ {On ■■ N >1) and denote by the complement of S: T," = (B^ : N >l). 

Definition 2.1 (Valley). The triple (W, 23,^) is a valley of depth 9 and attractor 
^ for the Markov process {rj^ '■t>0] if for every point rj = (77^ ■ N > 1) in W 

(VI) With overwhelming probability, the attractor^ is attained before the process 
leaves 23 ; 

lim P^n[T^<T^.] = 1; 

(V2) The law of T's,c /On under 'P^^n converges to a mean 1 exponential distri- 
bution, as N 00; 
(V3) For every S > 0, 



lim P„ 



= 



where A. — [An ■ N > 1) and An is the annulus Bn \ Wn ■ 

We refer to W as the well, and !B as the basin of the valley (W, 23,^). We 
present in Section [3] examples of Markov processes on finite state spaces and triples 
(W, 23, in which all conditions but one in the above definition hold. 

Condition (VI). The first condition guarantees that the process thermalizes in W 
before leaving the basin 23. We prove in Lemma [4.11 that conditions (VI), (V2) 
imply that the attractor ^ is reached from any point in the well W faster than On'- 

r 1 



lim sup P,, 



■T^>5 



. (VI') 



Conversely, this condition and (V2) warrant the validity of (VI). We may therefore 
replace (VI) by (VI') in the definition. 

Example 13 .21 illustrates the fact that conditions (V2), (V3) may hold while (VI) 
fails. In this example, with overwhelming probability, the process, starting from 
one state in the well W, leaves the basin 23 at an exponential time before hitting 
the attractor ^. 

Of course, the existence of an attractor is superfluous, as shown by Example 
3.8[ where we present a valley without an attractor. This requirement could be 
replaced by weaker requisites on the spectrum of the generator in the reversible 
case or on the total variation distance between the state of the process and the 
invariant measure restricted to the well W. Nevertheless, in several non trivial 
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examples, as in the case of condensed zero-range processes [31 14] which motivated 
this paper, attractors do exist. 

Condition (V2). The second condition asserts that the process leaves the basin at 
an exponential time of order given by the depth of the valley. Example 13 . 51 presents 
a situation in which conditions (VI), (V3) hold but not (V2) nor (VI'). There, 
the order of magnitude of the time needed for the process to reach S"^ from W 
depends on the starting point of W. 

Clearly, the depth of a valley is defined up to an equivalence relation: if 6' — {9'^ : 
N > 1) is another sequence of positive numbers such that limjv^oo(^Ar/^Ar) ~ 1; 
the valley has also depth 9' . Moreover, the depth of a valley depends on the basin. 
As we shall see in Example 13.31 two different valleys (W, 23,^), (W,^',^), with 
23 C 23', may have depths of different order. Finally, the depth has not an intrinsic 
character, in contrast with valleys, in the sense that it changes if we speed up or 
slow down the underlying Markov process. 

Condition (V3). The last condition requires the process starting from the well to 
spend a negligible amount of time in the part of the basin which does not belong 
to the well. 

We prove in Lemma 14.21 that we may replace condition (V3) by the assumption 
that for every point t] = {r}^ : > 1) in W and every t > 0, 



Condition (V3) is necessary, as we shall see in Example 13.11 to ensure that W 
is the well of the valley and not an evanescent set. The Markov process presented 
in this example fulfills conditions (VI), (V2) but not condition (V3). 

The definition of valley focus on paths of the Markov process starting from the 
well W. Nothing is imposed on the process starting from the annulus A, which 
may hide other wells, even deeper than the well W, as illustrated by example 13.71 
To rule out this eventuality, we replace condition (V3) by assumption (V3') which 
reads: 

For every (5 > 0, 



Fix ry in A^r and note that T-wus^ = r-wuS'^(^) !i 73=(A) almost surely. 
Therefore, it follows from condition (V3') that the process starting from A imme- 
diately reaches W U 23^: For every (5 > 0, 




(2.4) 



lim sup P,, — T'3<=(A)>(5 



. 



(V3') 



lim sup P^ — T-wu'B'' > S 
It also follows from conditions (V2), (V3') that 

/.min{t,0-^T^c} 



= 0. 



(2.5) 





(2.6) 



These remarks lead naturally to a more restrictive definition of valley. 
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Definition 2.2 (S- Valley). The triple (W,!B,^) is a S-valley of depth 9 and at- 
tractor ^ for the Markov process {rj^ : ^ > 0} if, for every point t] — [rj^ ■ N > 1) 
inW, assumptions (VI), (V2), (V3') are fulfilled. 

In Example 13. 71 we present a triple (W, S, ^) satisfying assumptions (VI), (V2), 
(V3) but not (V3'), (|2.5p and (|2.6p because A contains a well deeper than W. 

In many cases, it is possible to transfer from A to 'B'^ all points in A which 
do not reach immediately W U CB"^, in the sense of condition (|2.5[) . to obtain from 
a valley (W,®,!) satisfying conditions (VI), (V2), (V3) a new valley (W,S',|) 
satisfying conditions (VI), (V2), (V3'). We refer to Example lO 

We present in Example 13.41 a triple which satisfies conditions (VI), (V2), ()2.5|) 
but not (V3). In particular, the first three conditions do not imply (V3). In this 
example, there is a state in the annulus A^r which immediately jumps to the well 
Wn, but which is visited several times before leaving the basin Bj\j. 

2.2. Tunneling and Metastability. Given a sequence of Markov processes {77^ : 
t > 0} with values in Ejsf, we might observe a complex landscape of valleys with a 
wide variety of depths. We describe in this subsection the inter-valley dynamics. 

Fix a finite number of disjoint subsets . . . , k > 2, of En: £^ n £](^ = 0, 
X ^ y. Let S.N = ^x£s£-n ^^'^ I'^t An — En \£.n so that 

En = £]vU---U£^UA7v . 



Denote by "^n ■ £-n '-^ S = {1,2, . . . , k}, the projection given by 

^Niv) - 5].Tl{r?e£^} 

xes 

and let 

£:^ £jv \ £^ , £" = (£^ : iV > 1) and £" = (£^ : TV > 1) . 

For a subset A of En , let be the generalized inverse of the additive functional 
T/^ introduced in the beginning of this section: 

Sf{e.) sup{s>0:r/(e.) <i}. 

It is clear that < +00 for every t > if, and only if, T/^ — >+cx)as<^+cx3. 
To circumvent the case — 00, add an artificial point to the subset A. For any 
path e. e £'(IR+, En) starting at cq S A, denote by e;^ the trace of the path e. on 
the set A defined by ef = CgA if Sf < +00, and ef = D otherwise. Clearly, if 
ef^V for some t, then ef = 0, for every s > t. 

Denote by {r/f. " : t > 0} the £Ar U {i)}-valued Markov process obtained as the 
trace of {rjt : t > 0} on. En, and by {X^ ■ t > 0} the stochastic process defined 
by — ^'Ar(77f ") whenever r]^" G £jv and X^ = otherwise. Clearly, besides 
trivial cases, X!^ is not Markovian. 

Let 6 = {9n ■ N > 1) denote a sequence of positive numbers and, for each 
X & S, let = {^^ : > 1) be a point in £^. In order to describe the asymptotic 
behaviour of the Markov process on the time-scale 6 we use a Markov process 
{¥ X '■ X € S} defined on the canonical path space -D(IR-(-, S). 

Definition 2.3 (Tunneling). A sequence of Markov processes {rj^ : t > 0}, N > 1, 
on a countable state space E = {En '■ N > I) exhibits a tunneling behaviour on the 
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time-scale 6, with metastates {£^ : a; G 5}, metapoints {^^ : x € S} and asymptotic 
Markov dynamics {¥ x '■ x (z S} if, for each x £ S, 
(Ml) The point is an attractor on £^ in the sense that 
lim inf PJn < J = 1 : 

(M2) For every point rj — {rj^ : N > 1) in E,^ , the law of the speeded up process 

{X^^ : t > 0} under P^n converges to P ^ as N 1 oo; 
(M3) For every t > 0, 

= 0. 



lim sup / l{ij%^eAN}ds 

^+00 j,g£^ L Jq 



Let A denote the sequence (An N > 1) and consider the triple (£^, £^U A,^^) 
for a fixed x in S. Clearly, if x is not an absorbing state for the asymptotic Markov 
dynamics, the triple (£^,£^ U A,^^) is a valley of depth of the order of 6. In 
this case, it may happen that the triple (£^,£^ U A,^a;) is an inaccessible valley 
in the sense that once the process escapes from £^ it never returns to £^. This is 
illustrated in Example 1 3. 6 1 In contrast, if x is an absorbing state for the asymptotic 
Markov dynamics not much information is available on the triple (£^, £^ U A, ^^). 
Example 13.51 presents a Markov process which exhibits a tunneling behavior in 
which a triple is not a valley. In this example the triple contains a well of larger 
order depth than 9. 

Suppose that property (M2) is satisfied for a sequence of Markov processes and 
denote by 5'* C 5 the subset of non-absorbing states for {F^ : x € S}. For the 
states in S** we may replace requirement (M3) by property (V3) of valley, namely: 
For each x G 5', , 

Jim sup P^[^Tg,(A) >(5] = 0. (CI) 



Proposition 2.4. Assume that (M2) is fulfilled for a sequence of Markov processes 
{Vt : i > 0}, iV > 1. // (MS) IS satisfied for each x e S\S^ and if (CI) holds for 
any a; G 5*, then (M3) is in force for any x G S*. 

We arrive to the same conclusion in Proposition 12.41 if we assume instead that 
(CI) holds for every state x € S. This is the content of Lemma [4.71 Actually, for 
an absorbing state x, property (CI) is stronger than (M3) because in this case 
Ojj^Tg^ diverges. 

The definition of tunneling examines the inter-valley dynamics between wells 
with depths of the same order. It is far from a global description since it does 
not exclude the possibility that A contains a landscape of valleys of depths of 
larger order than 6. This situation is illustrated in Example 13.71 We have also 
just seen that if x is an absorbing state for the asymptotic Markov dynamics, the 
set £^ may also contain a landscape of valleys of larger order depth. In order to 
exclude these eventualities, we impose more restrictive conditions in the definition 
of metastability. We replace (Ml) by (Ml') to ensure that there are no wells in £^ 
of depth of order 9n if x is an absorbing point for the asymptotic Markov dynamics; 
and we replace (M3) by (M3') to avoid wells in A of depth of order 9^ or larger. 

Definition 2.5 (Metastability). A sequence of Markov processes {rj^ : t > 0}, 
N > 1, on a countable state space E — {Ejq N >1) exhibits a metastable behaviour 
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on the time-scale 0, with metastates {£^ ; x £ S*}, metapoints {^^ : x € S} and 
asymptotic Markov dynamics {¥ x : x G S}, if for each x & S, 

(Ml') The point is an attractor on £^ in the sense that for every S > 



(M2) For every point r/ — {rj^ : N > 1) in , the law of the speeded up process 

{X^^ : t > 0} under P^n converges to Px as N oo; 
(M3') For every t > 0, 



As for valleys, it follows from (M3') that A is evanescent in the sense that for 
every S > 0, 



Example 13.51 presents a Markov process which exhibits a tunneling behaviour 
and fulfills condition (M3') but violates assumption (Ml'). Example 13.71 presents 
a Markov process with the opposite properties. It fulfills conditions (Ml'), (M2), 
(M3) but violates assumption (M3'). This latter example is very instructive. It 
shows that the same Markov process may have distinct metastable behaviors at 
different time scales. This occurs when on one time scale there is an isolated point 
in the asymptotic Markov dynamics. In longer time scales this metastate is reached 
by other metastates, previous metastates coalesce in one larger metastate, and a 
new metastable picture emerges. 

We conclude this subsection observing that we may define metastability without 
referring to trace processes. Indeed, consider the S'-valued stochastic process 
defined as 



where a{t) :— sup{s < t : rj^ £ £.n}- Note that is well defined whenever rj^ 
starts from a point in S^r . 

Proposition 2.6. In condition (W12) of Definitions f^TM 1^.51 we may replace the 
stochastic process {X^^ ■ t > 0} by {X^^ : t > 0}. 

2.3. The positive recurrent case. The purpose of this subsection is to provide 
sufficient conditions to ensure tunneling. Assume from now on that the Markov 
process {rj^ : t > 0} is irreducible and positive recurrent, and denote by hn its 
unique invariant probability measure. It follows from these hypotheses that the 
holding rates A^(ry) = —RN{r],r]) are strictly positive, and that the discrete time 
Markov chain on which jumps from 7y to ^ at rate Rn{v^ is irreducible 

and recurrent. 

Furthermore, for every r] e En and A C En, T/^ diverges. Consequently, the 
trace of the Markov process {77^ : t > 0} on the set A, denoted by {ijf : t > 0}, 
is well defined and takes values in A. In fact, we prove in Proposition 16. II that the 
trace {rjf : t > 0} is an irreducible and positive recurrent Markov process with 
invariant probability measure equal to the measure /zat conditioned on the set A. 



lim sup PjT^ > SOn] = ; 




lim sup PJTe^ >S0n] = . 



(2.7) 
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Consider two sequences of sets W and S satisfying ()2.3|) . To keep notation simple, 
let An = Bn\ Wn, A = (A^ : N > 1), a.iid En = Wn U B%, E, ^ {En : N > 1). 
Denote by 

the transition rates of the Markov process {rjf" : t > 0}, the trace of {rjf : t > 0} 
on En- Let Rj^ : Wn ^ K+ be the rate at which the trace process jumps to B'^: 

and let rjv(W, 'B'^) be the average of i?]^ over Wn with respect to , the measure 
/iAT conditioned on Wn- 

^ (2.8) 

Next theorem presents sufficient conditions for W and H to be the well and the 
basin of a valley. 

Theorem 2.7. Assume that there exists a point ^ — (^^ ■ N > 1) in W such that 
for every point ij = {rj^ : N > 1) in W, 

lim [ / ' R^iv^) l{?7f e Wn} ds] = , (2.9) 

lim rA,(W,S")E„iv[T^(W)] = (2.10) 

AT— >oo 

and 

lim rAr(W,!B'=)E iv[rse(A)] = 0. (2.11) 

Then, (W,®,^) is a m^e?/ with depth 9 ^ {9n : N >1) where On = l/rA,(W, S^), 
iV > 1. 

Conditions (j2.9p and (I2.10|) clearly follow from the stronger condition 
lim sup{i?)^(77) : r] G Wn}'E„n[T^] = 0. 

N^OQ 

To state sufficient conditions for a tunneling behaviour, recall the notation in- 
troduced in Subsection 12. 21 Let R^ : En ^ En ^ K+ be the transition rates of the 
trace process {rjf" : t > 0}, let R^^ : Ef^ K+, x,y ^ S , x y, he the rate at 
which the trace process jumps to the set Ef^: 

R^^y{r,) := E R%{V,0, 

and let Rf^ : E'f^ — + R+, a; £ 5, be the rate at which it jumps to the set £^: R^ = 
J2y^x Rn^ ■ Observe that Rf^ coincides with Rj^ if {Wn,Bn) = (fi^f, E^ U An)- 

Let fi^ stand for the probability measure fiN conditioned on Denote by 
rN{E^ , E^) the /^^-expectation of i?^^: 

E fi^"WA*v(r7) 
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and by rAr(£^, £^) the /i^-expectation of so that 



To guarantee that the process {77^ : i > 0} exhibits a tunnehng behavior, we 
first require that each subset {t^ : x € S} satisfies conditions (|2.9p and (|2.10p : For 
each X & S, there exists a point = {^^ : N > 1) in E.^ such that 



hm E„i\ 



/ RU€)l{v^ eE,%}ds] = (C2) 
Jo ^ 



and 



hm rAr(£^,£^)E„iv[T^ (£^)] = (C3) 
for every point t] = (77^ : > 1) in £^. 

Theorem 2.8. Suppose (C2), (C3) and that there exists a sequence 6 = [6^ ■ 
N > 1) of positive numbers such that, for every pair x,y £ S, x ^ y, the following 
limit exists 

r{x,y) hm rjv(£", £^) • (HO) 

A— >oc 

Assume, furthermore, that (M3) is satisfied for each absorbing state x of the 
Markov process on S determined by the rates r and that (CI) holds for any non- 
absorbing state. Then, the sequence of Markov processes {rj^ : t > 0}, N > 1, 
exhibits a tunneling behaviour on the time-scale 9, with metastates {£^ : x £ S}, 
metapoints {^^ : x £ S} and asymptotic Markov dynamics characterized by the 
rates r{x, y), x,y £ S. 

Notice that in the previous theorem we might get 

r{x,xo) = and r{xo,x) > 

xGSViaJo} xeS\{xo} 

for some Xq £ 5*. In this case, the triple {E.^",E.^° U A,^^^) turns out to be an 
inaccessible valley, as it is illustrated in Example 13.61 even tough it has the same 
depth than all the other wells involved in the tunneling. 

2.4. The reversible case, potential theory. In addition to the positive recur- 
rent assumption, let us now further assume that fiN is a reversible probability 
measure. In this case, we may list simple conditions, all of them expressed in 
terms of the capacities and the reversible measure (in, which ensure the existence 
of valleys and the tunneling behaviour. 

As we have already seen, we need good estimates for the mean of entry times. 
In the reversible case, the mean of an entry time has a simple expression involving 
capacities, which are defined as follows. For two disjoint subsets A, B of E'at define 

Cn{A, B) := {/ £ L^ifiN) : f{v) = 1 V r; e A and f{0 = V ^ £ B} ■ 

Let stand for the scalar product in i^(/ijv). Denote by the Dirichlct 

form associated to the generator L^: 

DnU) ■■= {-LNfJ)>., , 
for every / in L'^{pn)- An elementary computation shows that 

DNif) = I Mv)RNiv,0{f{0~f{v)V- 
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The capacity of two disjoint subsets A, B of En is defined as 

capjv(A, B) inf { : / e Cat (A, B) } . 

In the reversible context, the expressions appearing in Theorems 12.71 and 12.81 
can be computed by using capacities. Denote by f^g : K the function in 

Cn{A,B) defined as 

In addition, for two points ^ = (^^ : TV > 1) and (77^ : iV > 1) in W, 77^ ^ , 
N>1, set fNiv,& = and capjv(J7,l) = cap^({77^}, {^^}). 

Consider two sequences of sets W and S satisfying (|2.3p and recall the notation 
introduced in the previous subsection. By (|6.16p . 



RJ^1{Wn}, fN{v,^))^^. 



and, by Lemma 16.71 



J capjv(J7,l) 

Mw,a3^) = £^Hiv(^. (2.14) 

In the last identity cap^(W, S^) := cap^(M^Ar, S^) and ^Ar(W) ^n{Wn)- The 
previous relations can be used to check conditions (|2.9p and (|2.10p in Theorem 12. 71 
as well as assumptions (C2) and (C3) in Theorem [ 



Furthermore, since < /jv(^,^) < 1, and since, by (|2.8p . {R^ 1{Wn}) ^^M 
^Ar(W)rAr(W,S=), by (Eia, 



l-Tt 

E 



Jo 

and rAr(W,!B")E„iv[r^(W)] < 



^ cap^(W,a3^) 
capjv(^) 
capjv(W,B^) 



capjv(l) 

where cap^(^) — inf{cap^(77, ^^) : 77 e W \ {^^}}. Hence, conditions (|2.9p and 
p.lOp follow from the stronger condition 

cap^(W,S'=) 

lim ' , ' = 0. 2.15) 

Af-*oo capjv(4) 

Theorem 2.9. Assume that \2.15]) holds for some point ^ — (^^ : -/V > 1) m W 

and that 

lim ^^(^;\^^) ^0. (2.16) 

Then, for all points C, in W, (W, S,^) is a valley of depth /iAr('W)/capjvCW, S^), 
N>1. 

Assumption p.lSp is also powerful in the context of tunneling. Recall the nota- 
tion introduced at the beginning of Subsection 12.21 

Theorem 2.10. Suppose that for each x S , there exists a point — {^^ : N > 
1) m £^ such that 

lim ^"p^(^;;^;) = . (HI) 
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Suppose, furthermore, that (HO) holds for some 9 = [On '■ N >1), that (M3) holds 
for each absorbing state of the Markov dynamics on S determined by the rates r 
and that 

linii^^O, (H2) 

for each non-absorbing state x. Then, the sequence of Markov processes {rj^ : t > 
0}, > 1, exhibits a tunneling behaviour on the time-scale 9, with metastates {£^ : 
X g S}, metapoints {^^ : x ^ S} and asymptotic Markov dynamics characterized 
by the rates r[x, y), x,y d S. 

Remark 2.11. In the previous theorem, we may replace condition (M3) for ab- 
sorbing states and condition (H2) for non-absorbing states by the assumption 

lim '—^ = (H2') 

for all states x. 

Note that condition (H2) and (H2') are equivalent for non-absorbing states if 
(HO) holds. This latter condition can be expressed in terms of capacities since, by 
Lemma /xjv(£^)rjv(£^, £^) can be written as 

i{ cap^(£^ £") + capJv(£^ £^) - capjv(£" U £'^ £ \ (£" U £'')) } 

for every x,y £ S, x y. 

One of the main steps in the proof of metastability is the replacement result pre- 
sented in Lemma l6.4l and in Corollarv l6.5l This statement proposes a mathematical 
formulation of the notion of thermalization by identifying this phenomenon with 
the possibility of replacing the time integral of a function by the time integral of its 
conditional expectation with respect to the cr-algebra generated by the metastable 
states. The existence of attractors allows a simple estimate, presented in Corollary 
16.51 which plays a key role in all proofs. 

The propositions stated above are proved in Section |4l while the theorems and 
the remark are proved in Section O 

3. Some examples 

We present in this section some examples to justify the definitions of the previous 
section and to illustrate some unexpected phenomena which may occur. 

We start with a general remark concerning valleys on fixed state spaces. Consider 
a sequence of Markov processes {rj^ ■ t > 0} on some given countable space E with 
generator Ljy described by (|2.ip . Denote by Xn{v) — Rn{VtO the rate at 

which the process leaves the state 77. Clearly, the triple ({??}, {?]}, ??) is a well of 
depth XN{ri)~^ in the sense of Definition 12.11 

The first example highlights the role of condition (V3) in preventing some evanes- 
cent sets to be called wells. 

Example 3.1. Consider the sequence of Markov processes {yyf ■.t>0}onE = 
{ — 1,0, 1} with rates given by 

Rn{-1,0) = i?Ar(l,0) = N, Rn{0,-1) = -Riv(0,l) = 1, 

and RnU, k) ~ otherwise. 
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Obviously, we do not wish the triple ({ — 1}, { — 1,0}, —1) to be a valley. Never- 
theless, this triple satisfies conditions (VI) and (V2) of Definition 12.11 The first 
one is satisfied by default. To check the second one, note that starting from —1 

M 

where {Sj : j > 1}, {Tj : j > 1} are independent sequences of i.i.d. exponential 
random variables of parameter N, 1, respectively, and M is geometric random vari- 
able of parameter 1/2, independent of the sequences. Hence, (1/2) T^c converges 
in distribution, as iV f oo, to a mean 1 exponential random variable. 

It is condition (V3) which prevents the triple ({ — 1}, { — 1, 0}, —1) to be a valley 
since the time spent at before reaching { — 1, 1} is a mean 1/2 exponential random 
variable. □ 

Next example illustrates the fact that conditions (V2), (V3) may hold while 
(VI) fails. 

Example 3.2. Consider the Markov process on {0, 1, 2, 3} with rates given by 
i?^(l,0) = i?w(2,3) = l-(l/iV), i?w(l,2) = i?Ar(2,l) = 1/N, 
Rn{0,0) = a, i?Ar(3,3) = b 

for some a, b > 0, and _Rjv(i, j) = otherwise. 

Consider the tripe ({1, 2}, {1, 2}, 1). It is clear that condition (VI) does not 
hold since the process starting from 2 reaches 'B'^ — {0, 3} before hitting 1 with 
probability 1 — (1/iV). Condition (V2) is fulfilled for 6* = 1 because Ts<: converges to 
a mean one exponential time, independently from the starting point, and condition 
(V3) is in force by default. □ 

The third example illustrates the fact that the depth of a valley depends on the 
basin. 

Example 3.3. Consider the sequence of Markov processes 

{rit ■■t>G) on E = 

{ — 1,0, 1} with rates given by 

i?Ar(-l,0) = i?Ar(l,0) = 1, i?jv(0,-l) = i?jv(0,l) = N , 

and RnU, k) = otherwise. 

By the observation of the beginning of this section, the triple ({ — 1}, { — 1}, —1) is 
a valley of depth 1. On the other hand, the triple ({ — 1}, { — 1, 0}, —1) is a valley of 
depth 2. Condition (VI) is satisfied by default, and condition (V2) can be verified 
by representing the time needed to reach 'B'^ as a geometric sum of independent 
exponential random variables, as in Example 13.11 Requirement (V3) is readily 
checked. □ 

Next example shows that conditions (VI), (V2) and (|2.5p do not imply (V3). 

Example 3.4. Consider the sequence of Markov processes {ry^ ■ t > 0} on E = 
{1,2,3} with rates given by i?Ar(l,2) = N, i?iv(2,l) = iV - 1, i?(2,3) 1 and 
R]si(i,j) = otherwise. 
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Let ^ = 1, = {1} and B = {1,2}. Condition (VI) is fulfilled by default. 
Condition (V2) is easily checked for 9n = 2. In fact, the hitting time of 3 
starting from 1 can be written as J2i<j<Mi^3~^'^j}^ where {Sj : j > 1}, {Tj : j > 1} 
are independent sequences of i.i.d. mean 1/N exponential random variables and M 
is a geometric random variable of parameter independent of both sequences. 
It follows from this representation that /2 converges in distribution to a mean 1 
exponential random variable. 

For similar reasons, conditions (V3) fails: With the notation just introduced, 
starting from 1, the time spent at state 2 before hitting 3, denoted in Section [2] by 
T^(2), converges to a mean 1 exponential random variable. 

Condition p.5|) . however, is in force, since the hitting time of the set {1,3} 
starting from 2 is of order 1/A^. □ 

The fifth example shows that metastates might not be wells of valleys. It presents 
also a triple which fulfills condition (VI), (V3) but not (V2) nor (VI'). 

Example 3.5. Consider the sequence of Markov processes 

{rit ■■t>Q} on E = 

{0, 1,2} with rates given by 

Rn{1,0) = ^-1, i?Ar(l,2) = 1, Rn{2,1) = N-^ , Rn{0,1) = N'^ , 
and RnU, k) = otherwise. 

The triple ({1, 2}, {1, 2}, 2) is not a well because condition (VI) is violated. With 
overwhelming probability the process starting from 1 leaves the set {1,2} before 
reaching 2. The triple ({1, 2}, {1, 2}, 1) is not a well either. While conditions (VI), 
(V3) are clearly satisfied, it is not difficult to show that condition (V2) is violated. 
In fact, starting from 1, T^c converges to a mean one exponential random variable, 
while starting from 2, N^^T-S" converges to a mean one exponential random vari- 
able. It is also clear that condition (VI') fails in this case since on the scale of 
order 1 the process starting from 2 never reaches 1. 

At the scale iV~^ the process exhibits a tunneling behaviour, as described in 
Definition 12.31 with metastates = {0} and £^ = {1,2}, ^2 — 1; ^'^d asymptotic 
Markov dynamics characterized by the rates r(l, 2) = 1, r{2, 1) — 0. It does not 
exhibit a metastable behaviour, as described in Definition 12. 5[ because condition 
(Ml') is violated. Starting from state 2 S the process never reaches the attrac- 
tor 1 in the time scale N~'^. We have also here an example of an absorbing set for 
the asymptotic dynamics which is not a valley due to the existence of the well {2} 
in the the set of depth N > N'"^. □ 

Next example shows that there might exist inaccessible valleys. 

Example 3.6. Consider the sequence of Markov processes iv? : t >0} on E = 
{1, . . . , 5} with rates given by 

RN{j,k) — 1 if j is even, k odd and |j — fc| = 1 , 

Rn{1,2) = i?Ar(3,4) = i?jv(5,4) = N'^ , i?w(3,2) = N'^ , 

RN{i,k) — otherwise. 

The triples ({1}, {1, 2}, 1), ({3}, {3, 4}, 3), ({5}, {4, 5}, 5) are valleys of depth 
2N . Moreover, at the time scale N the process exhibits a metastable behaviour, 
as described in Definition 12. 5| with metastates £^ = {1}, £^ = {3}, £^ = {5} 
and asymptotic Markov dynamics characterized by the rates r(l,2) = '^(2,3) — 
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r(3, 2) ~ 1/2, r{i,j) — 0, otherwise. Note that the metastate £^ is inaccessible in 
the sense that r(2, 1) + r(3, 1) = 0. This means that in the time scale N the process 
starting from 1 eventually leaves this state, never to return. □ 

The penultimate example, very instructive, shows that different phenomena may 
be observed on different scales. It also highlights the role of conditions (V3), (V3'). 

Example 3.7. Consider the sequence of Markov processes iv^ : t >0} on E ^ 
{1, . . . , 5} with rates given by 

RnUj k) — I if j is even, k odd and |j ~ A:| = 1 , 

i?Ar(l,2) - N-^ , i?^v(3,2) = , i?^(3,4) = i?jv(5,4) = N'' , 

RN{j,k) — otherwise. 

A simple computation shows that the measure on E given by niN^l) — A^^, 
TOAr(2) = 1, mAr(3) = , mAr(4) = N^, mAr(5) = is reversible for the Markov 
process. We leave to the reader to check that ({3}, {3, 4}, 3), ({5}, {4, 5}, 5) are val- 
leys of depth 2N, and that ({!}, {1, 2}, 1), ({3, 4, 5}, {3, 4, 5}, 3), ({3, 4, 5}, {2, 3, 4, 
5}, 3) are valleys of depth 2iV^, 2A^'^, 4A^'^, respectively. The presence of valleys of 
different depths leads to diverse tunneling behaviors at different time scales. 

This example illustrates that we may have valleys satisfying conditions (VI), 
(V2) and (V3), but not (V3') and (EH). This is the case of the triple ({3}, {1, 2, 3, 
4}, 3). The latter conditions are violated because the annulus {1,2,4} contains 
the valley ({1}, {1, 2}, 1) of depth 2N^, larger than 27V which is the depth of 
({3}, {3, 4}, 3). On the scale N, the process starting from 1 never reaches 3 with 
positive probability. However, condition (V3) holds because on the scale N the 
process starting from 3 never reaches {1,2}. 

Note that transferring the points 1, 2 from A to S"^, we transform the the valley 
({3}, {1, 2, 3, 4}, 3) in the S-valley ({3}, {3, 4}, 3). 

At the scale A^ one observes a tunneling between £^ = {3} and £^ = {5}, 
characterized by the asymptotic Markov rates ^(l, 2) — r{2, 1) = 1/2. Assumption 
(M3') is not satisfied because the set An contains a well of depth larger than the 
depth of the metastates. However, this well is never visited if the process starts 
from one of the metastates. 

To turn the tunneling behavior into a metastable one, we may add the metastate 
E.^ = {1} and show that at scale A^, the process exhibits a metastable behaviour 
with metastates £^ — {3}, £^ — {5}, £'^ ~ {1} and asymptotic Markov dynamics 
characterized by the rates ?'(1,2) = r(2, 1) — 1/2, r{i,j) = 0, otherwise. Observe 
that an isolated state has appeared in the asymptotic dynamics. 

At scale A^^, the metastates £^ — {3}, £^ = {5} coalesce into one deeper well. 
In this scale the process exhibits the metastable behaviour with metastates E.^ — 
{1}, £^ = {3,4,5}, and asymptotic Markov dynamics characterized by the rates 
r(l,2) — 1/2, r(2, 1) — 0. Note that we have here an absorbing asymptotic state 
and that {3, 4, 5} is not the well of a valley of depth of order A^^, but the well of a 
valley of depth of order A^'^. □ 

The last example shows that the existence of an attractor is superfluous in the 
definition of a valley. Denote by En = [l/NZY U {l/NlY the union of two d- 
dimensional torii of length A^ and denote by x G (Z/A^Z)"*, j — ±1, the 

elements of En- 
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Example 3.8. Consider the sequence of Markov processes {?7f : t > 0} on En 
with rate jumps given by 

RN{{x,j),{x',j)) = ^l{\x - x'\ = 1} , RN{{x,j),{x,-j)) ^ , 

for some rate On such that iV^ « On « N"^ , and RN{{x,i), {y,j)) — otherwise. 

It is well known that the spectral gap of the symmetric simple random walk on 
the torus (Z/iVZ)'' is of order N~'^. The evolution of the process rj^ is therefore 
quite clear. In a time scale of order iV^, the process thermalizes in the torus where 
it started from, and after an exponential time of order ^^r it jumps to the other 
torus, replicating there the same qualitative behavior. 

Hence, each torus satisfies all reasonable conditions to be qualified as a valley of 
depth On- Nevertheless, there is no attractor in this example since a specific state 
is visited by the symmetric simple random walk only in the scale N"^. □ 



4. Valleys and metastability 

In this section we prove some results on valleys and on tunneling. The first lemma 
states that we may replace condition (VI) by condition (VI') in the definition of 
a valley. 

Lemma 4.1. In Definition \2.1\ condition (VI) may be replaced by condition (VI'). 

Proof. Let us denote by 8t := 0f , t > 0, the time-shift operators on the path 
space D{R+,En). Let (W, S, ^) be a valley of depth 6 = {On : N >1). Fix a point 
rj = {rjN ■ N > 1) in W as the starting point. Consider the pair of random variables 
Tg, T^c o Qt^, which are independent by the strong Markov property. According 
to assumption (VI), the event {T^ < Tsc} has asymptotic probability equal to 
one. On this event + T^c o Qx^^ = . Since, by assumption (VI), O^^T-b" 
converges to a mean one exponential random variable, Ojj^{T^ + T^c o Qt^} also 
converges to a mean one exponential random variable. 

Suppose by contradiction that there exist 6, e > such that 



lim sup P,,„ 

N^cx> 



(4.1) 



By assumptions (VI), (V2), 0^^(rsc o GtJ converges to a mean one exponential 
random variable and, by (|4.ip . O^^T^^ > S with strictly positive probability. In 
particular, 0~^{T^+T'b.= oQt^} may not converge to an exponential random variable, 
in contradiction with the conclusion reached above. 

Conversely, the event {T^ < Tsc} contains the event {T^ < (5^^^^} n {Tsc > 60n} 
for every S > 0. By assumptions (VI'), (V2), the Prjjv" probability of this event 
converges to 1 as | c« and then S i 0. This concludes the proof of the lemma. □ 

The second result examines the assumptions (V3) and (V3') in the definition of 
valleys. 

Lemma 4.2. In Definition \2.1[ assumption (V3) may be replaced by (j2.4p . and in 
Definition \8.2\ assumption (V3') may be replaced by p.6p . 
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Proof. The time integral in ()2.4|) is bounded above by min {t,0jviT3o(A)}. There- 
fore, ([2?4l) follows from (V3). 

Conversely, the time integral in (113) is bounded below by 6'^^T3c(A)l{r23c < 
i6'jv}. This expression is itself bounded below by min{6'jY^r3<:(A), a}l{TcBc < i^Ar} 
for every a > 0. Therefore, 

l-min{t,e-^T^c} 

min{0-iTsc(A),a} < / l{r/,e„ e An} ds + al{T^a > te^} 

JQ 

for every a > 0. Fix a point t] — {i]^ : iV > 1) in W. By (|2.4p . the expectation 
with respect to P,,^jv of the first term on the right hand side vanishes as N ^ oo 
for every t > 0. By (V2), the expectation with respect to P^w of the second term 
vanishes as t and then i f oo. Therefore, for every a > 



lim E„^ 



min{0^iT3c(A),a} 



= 



This proves (V3). 

In the same way we prove that we may substitute assumption (V3') by (j2.6p in 
Definition 12.21 This concludes the proof of the lemma. □ 

Next lemma is needed in the proof of Proposition l4.41 one of the main results of 
this section. 

Lemma 4.3. Consider a subset A = {A]y : N > 1) of {Ejy : N > 1). Assume that 
there exists t > and e < 1 such that 

limsup sup Pri[TA>t9N] < e. (4.2) 
Then, sup^g^y^^ E,JTa(W)] < [t/{l — e)](?jv for every N sufficiently large and 

lim limsup sup eJo^^TaCW) HTaCW) > K0n}\ = 0. (4.3) 

K^oo N^oo jjSVKjv L ^ 

Proof. The proof is a simple consequence of the strong Markov property and as- 
sumption (|4.2p . Consider the sequence of stopping times {Ik : > 1}, {Jk '■ k >!} 
defined as follows. Ii =0, Ji — tO^, 

4+1 = inf {t > Jk : ryf e Wat} , 4+i = 4+i + i^iv , fc > 1 , 

with the convention that Jk = 4+i = cxd if 7^ = cxd for some fc > 1. Let M be the 
first time interval [4, 4] in which the process visits A]^: 

M = min {fc > 1 : rj^ ^ An for some t £ [Ik, Jk] or = oo} . 

Clearly, T/i(W) < tO^M . On the other hand, for N sufficiently large, by definition 
of the stopping times {4 '■ k > 1} and by assumption (|4.2p . M is stochastically 
dominated by a random variable M' with geometric distribution given by P[M' = 
fc] = (1 — e)e'^~^, fc > 1. This concludes the proof of the lemma. □ 

Next proposition gives an equivalent definition of a valley with attractor. 

Proposition 4.4. Assume that (W, S,^) is a valley of depth 6 and attractor ^. 
Then, for any point t] = (77^ ■ N > 1) in W, 
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(i) The hitting time of the attractor ^ is negligible with respect to the escape 
time from the basin 23 in the sense that 

E^«[T3e(W)] 

(ii) Under P^n, the law of the random variable T2c(W)/E,jiv [r3c(W)] con- 
verges to a mean-one exponential distribution; 

(iii) For every 5 > Q, 

Ts<=(A) 



lim P,- 



> 5 



= 0. 



E„«[Tsc(W)] 

Moreover, the sequences On anc? E,jJv [T3c(W)] are asymptotically equivalent in the 
sense that limjv^oo ^'jv^E^n [T^c (W)] = 1, 

Conversely, if (W,'B,^) is a triple satisfying p.3[) for which (i) - (iii) hold, then 
for any point t] = {rj^ ■ N > 1) in W, the sequence E,^n [T3c(W)] is asymptotically 
equivalent to E^n [Tsc(W)]; 



lim 



E„«[ra3.(W)] 



AT^oo E^jv [Ts<= (W)] 
and (Wj^,^) is a valley of depth 0, where On ~ E^n [Tcb^CW)]. 

It is implicit in the statement of this proposition that the time spent in the well 
W before leaving the basin S, T3c(W), has finite expectation with respect to any 
P^N for sufficiently large N , as well as the time spent in the well W before reaching 
the attractor |, T|(W). 

Proof of Proposition \4-4\ Assume that (W, S, ^) is a valley of depth 9 and attractor 
^. We first claim that 

lim sup -EJOn^ T^CW)] = 0. (4.4) 

This assertion follows from (VI') and the previous lemma with A = {£}, t = S, 
e = 1/2. 

Fix a point rj = {rj^ : > 1) in W. We claim that 

lim E„N[e]^'T^.{W)] = 1. (4.5) 

Three ingredients are needed to prove this result. The convergence of 61^^ Tsc to a 
mean one exponential random variable, a bound on E^« [e~^ rsc(W)] provided by 
the previous lemma, and the fact that the process does not spend too much time 
in A. 

We start with the proof of the lower bound. Fix S > 0, t > 0. On the set 
{rsc(A) < SOn}, we have that T-s^W) > T-b^ - S9n- Therefore, 

T3o(W) > -^SOn + Tn.llT-B^A) <dON} . 

Replacing T^^ by min{rsc,t0A} we obtain the estimate 

Tsc(W) > -S9n - t9Nl{T^c{A) > SOn} + min{rsc, t6'Ar} 

which holds for all 5 > 0, t > 0. 

By (V3), the expectation with respect to P^w of the second term on the right 
hand side divided by 9n vanishes as t oo for any fixed S > 0, t > 0. By (V2), the 
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expectation with respect to P,jiv of the third term on the right hand side divided 
by On converges to 1 as iV t cx3 and then t ] oo. Therefore, 

liininfE^iv[6l^irsc(W)] > 1. 

The proof of the upper bound is simpler. For every yl > 0, 

E„«[Ts.(W)] < E„«[min{Ts.,A0^v}] + [r25c(W)l{r3e(W) > A^at}] . 

By (V2), the first term on the right hand side divided by Ojsi converges to 1 as 
N ] oo and then A t oo. By (V2), (g^]) holds with = e = 1/2 and some 
t < oo. Therefore, by (|4.3p . the second term divided by 9^ vanishes as t oo and 
then ^ t oo. This concludes the proof of (|4.5p . 

Assertion (i) follows from (I4.4|l and (14. 5|) . and assertion (iii) from (V3) and (I4.5|l . 
Finally, T^c{W) ^ T^c - T-B'^i^)- By (V2), e^^j'^T^o converges in distribution to 
a mean one exponential random variable, and, by (V3), 0^^T3c(A) converges to 
in probability. Assertion (ii) follows from these facts and from (|4.5|) . The final 
claim of the first part of the proposition has been proved in 



To prove the converse, suppose that conditions (i) - (iii) hold. We first prove 
that (VI), (V2), (V3) are in force with replaced by the sequence 0{rj^) — 
E^iv [r3c(W)], which depends on the point r] — [rj^ N >l). In this case, condition 
(V3) corresponds to (iii). To prove (V2), note that Ts<= = Ts<=(W) + Ts<=(A). By 
(ii), 0(?7^)^^T3c('W) converges in distribution to a mean one exponential random 
variable and, by (iii), 6'(7/^)~^r3c (A) vanishes in probability. Therefore, (V2) 
holds. Finally, on the set {T^ < Tsc}, = T^ClV) +r^(A) and ^^(A) < r33c(A). 
By (i) and (iii), 6'(?7^)"ir^(W) and 6'(7?^)-iTsc(A) vanish in probability as T oo. 
On the other hand, by (V2), already proved, d{r]^)'^^T's<: converges in distribution 
to a mean one exponential variable. This proves (VI). 

It remains to show that the sequences 0{ri^) = E^n [T^cCW)] and E^n [T^c (W)] 
are asymptotically equivalent in the sense that their ratio converges to 1. 

By (ii) and Lemma [4.3[ the sequence 6'(77^)^^T2c (W) is uniformly integrable 
with respect to P,,iv. Therefore, by (VI), 



r3c(w)i{rg <Tsc} 



= 1 



By the strong Markov property and the explicit form of TscCVV), the expectation 
is equal to 



1 

xp 



1 



By (i), the first term vanishes as iV | 00. Since by (VI) P^N[r^ < T^c] converges 
to 1, E^jv [Ts<:("W)] and E^n [TscCW)] are asymptotically equivalent. This concludes 
the proof of the proposition. □ 



We conclude this section with the proofs of Propositions l2.4l and l2.6l Let us first 
fix a metric in the path space D(M+, S'U{t)}) which induces the Skorohod topology. 
In what follows, we identify the point 5 with G Z so that S'U{J)} is a metric space 
with the metric induced by Z. 

For each integer m > 1, let Am denote the class of strictly increasing, continuous 
mappings of [0,to] onto itself. If A G A„i, then Ao = and A„i = m. In addition, 
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consider the function 

{1 if < < m - 1 , 

m — t iim— l<t<m, 
if t > TO . 

For any integer to > 1 and e, e G D(M+, S U {O}), define dm{e, e) to be the infimum 
of those positive e for which there exists in A™ a A satisfying 

sup I At — 1 1 < e 

tG[0,m] 

and 

sup |5m(Af)eAt -gm(Oet I < e- 

tG[0,m] 

FinaUy, we define the metric in Z?(R+, S U {5}) by 

oc 

d{e,e) - ^2-"(lAd„(e,e)). 

m— 1 

This metric induces the Skorohod topology in the path space D(M.^,S U {d}) (cf. 

0)- 

For any path e £ D{R^,S U {5}) denote by (T„(e) : n > 0) the sequence of 
jumping times of e: Set To(e) = and, for n > 1, we define T„(e) as 

T„(e) := inf{t > T„_i(e) : et ^ eT-„_j(e)} , 

with the convention that t„ = oo if r„_i = oo and, as usual, inf = +oo. 

Proposition 12.61 is a consequence of the following result. 

Proposition 4.5. Suppose that {rjf : t > 0}, > 1, satisfies (M3) for any x ^ S. 

Then, for any x ^ S and point t] = {rj^ ■ N > 1) in , 

lim E„iv[d(X^,X^)] = . 

Proof. Fix arbitrary integers to > 1 and > 1. To keep notation simple, set 
Tn '■— Tn{X^) and f„ := t„(A'^), n > 0. Define the random variables 

n := sup{j > : fj < to} 

and 

T{X^) := Tn+i A TO . 

In Lemma l4!6l below we show that Pj^w-a.s., 

< 1^1 max{f„-r„ ; m-T{X^)} . (4.6) 
To estimate the right hand side in (j4.6p . observe that 

'rt 'n — -'fjj ^ -'m i 

where 7^'^" is the time spent by {r]f : t > 0} in Ajv in the time interval [0, i], 
introduced in (|2.2p . On the other hand, in the case t^+i < m, m — T(Ar^) can be 
written as to — {fn + [t„+i — Tn]} + [rn — Tn]. Since t„ — r,i is the time spent by 
iVt' ■ t > 0} in Aat in the time interval [0, fn] and to — {fn + [Tn+i — Tn]} is the 
time spent in Ajv in the time interval [fn , to] , 

TO - T{X'^) < r^" . 
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Therefore, by ()4.6|) we have just shown that 

d{X^,X^) < ^ 2^"(1 A |5|7;^") . 

m— 1 

The desired result follows from this estimate and property (M3). □ 

Lemma 4.6. For any integers N,m > 1, 114 -(^ holds P^^n -almost surely. 

Proof. Fix two integers N,m > 1. All assertions in what follows must be under- 
stood in the P^w-a.s. sense. Recall the notation introduced in the previous lemma. 

Let us list some evident properties of X'^ and X^: First notice that for all 
< j < n — 1, we have t,+i — ij > Tj+i — Tj and Xf = X^ for {s,t) G 
[Tj,Tj+i[x[fj,Tj+i[. Furthermore, Tn < T{X^) < m, ^ and = X^ for 
(s,t) e [T„,T(X^)[x[fn,m[. 

In particular, since fn < to, we may choose e > small enough such that 
Ta < T{X^) — e and fn < to — e. Now, let A e A™ be given by: Xf. = r, , for j < n, 
\m-t — T{X^) — e. Am — to and we complete A on [0, to] by linear interpolation. 
Then, 

sup |At — <| < maxjfn — t^, to — T{X^)} . 
te[o,m] 

Moreover, since At <t, < t < m, 

sup \gmi>^t)Xx^ ~ gmit)X^\ < \S\ sup |5m(At) - gm(t)| 

te[0,m-e] te[0,m-c] 

< IS*! sup \Xt-t\ 

iG[m — l.m — e] 

and 

sup |g„(At)X^-.g™(i)^f I < \S\ sup ( |<7™(At) - + 2|g,„(i)| ) 

£G [m — e,m] t£ [m — €,m] 

< IS"! sup |At - t| + 2Ke . 

Since e may be taken arbitrary small, the claim is proved. □ 

We now turn to the poof of Proposition 12.41 For every e G D{M.+ ,S U {f}), 
denote by Jt(e) the number of jumps up to time t: 

Jt{e) sup{j >0:Tj{e) <t} . 

Proof of Proposition \2.4] Fix an arbitrary non-absorbing state G 5* for the 
Markov process {Px ■ x G S}, a point t] = {rj^ : > 1) in and a time 
t > 0. It suffices to show that E^jv [T/^"] ^ as N ^ oo. 
For any integer AT > 1, 

T;^" < l{JtiX^) >K}t+ l{JtiX^) < K}%^« , (4.7) 

P^N-almost surely. The subset {Jt > K} C D{M.+ ,S U {t)}) is closed for the 
Skorohod topology. Therefore, by property (M2), 

limsupP^iv[Jt(X^) > a:] < limsupP^iv[Jt(A:^) > AT] < Vx'[Jt > K] . 

N—>oo N^oc 

The right hand side vanishes as AT j cx). From this and ()4.7p . it follows that 
limsupE,,iv[7;'^"] < limsuplimsupE^iv[l{Jt(X^) < ATIT^"^"] . 
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In consequence, in order to conclude the proof it is enough to show that 

hm E„jv[l{Jt(X^) ^ijTj-^"] =0, Vi>0. (4.8) 

N—^OG 

Fix some integer j > 0. To keep notation simple, denote Jt ■= Jt{X^) and let 
(jn : n > 0) stand for the jumping times of . Recall that we denote by 5* the 
set of non-absorbing states for {V^ : x G 5} and set £^ = yJxeS,^%i- On the event 
{Jt = i} let us define 

/ := ini{0<j<i:X?^^S\S^], 
so that I — Qoii and only if Xs G 5**, for all < s < t. On the one hand, P,jiv— a.s., 

l{Jt = z ; / = oo} T,^" < l{Jt = t ; I ^ oo}J2 l{v^eAN}ds 

1=1 -^^^-1 



3 

Thus, applying the strong Markov property we get 

E,,iv [l{Jt = i ; / = cx)}7^'^"] < [i + l) sup sup [t A Tg^A)] . 

The right hand side vanishes as TV ^ by assumption (CI) for the non-absorbing 
states. On the other hand, for any < i < i we have that, P^n— a.s., on the event 

{it = = 

< j2 T'^* i{,7f e A^} ds + l{,7f e A^} ds . 

By applying the strong Markov processes as before, we show that E^n [1{ Jt — 
i; I = £} Tf^"] is bounded above by 

i sup sup E^[iAr£.(A)] + sup E^[7;^] . 

As iV I oo, the first term vanishes as before while the second one vanishes by 
assumption (M3) for absorbing states. This concludes the proof. □ 



The same proof yields the following version of Proposition 12.41 which does not 
distinguish between absorbing and non-absorbing states. 



Lemma 4.7. Assume that (M2) is fulfilled for a sequence of Markov processes 
{Vt' ■ ^ ^ 0}, A'^ > 1. Then, condition (M3) is satisfied if for each x in S, 

1 

0^^ 



lim sup P^ 



-Tg^A) > 6 



0. 



Proof. The proof is simpler than the previous one. We do not need to introduce 
the variable /. We estimate as in the case / = oo to get that 

E^«[l{Jt =i}7;^"] < (i-M) sup sup E^[t AT£,(A)] . 
This expression vanishes as t c« by assumption. □ 
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5. Proof of the main theorems 

We prove in this section the main resuhs of the article. The proofs rely on some 
results on recurrent Markov processes presented in Section [S] 

Proof of Theorem 12.71 Next statement plays a central role in the proof of The- 
orem [1?71 

Proposition 5.1. Consider two sequences of sets W and H satisfying (|2.3p . As- 
sume that there exists a point ^ = (^^ : > 1) m W such that for every point 
T] = [r]^ : N > I) in W (HJ]) and ^^JO\i hold. Then, condition (VI) is in force. 
Moreover, the law of rN(W,'B'^) Ts^CW) under P^n converges to a mean-one ex- 
ponential distribution, as N ^ oo, and 

lim rN(W,S")E„iv[T3e(W)] = 1 (5.1) 

for any point rj — (rj^ '■ N > 1) inW 

The proof of this proposition is divided in several lemmas. Recall that En = 
Wn U B^j, N > 1, a,nd that {77^" : t > 0} stands for the trace of {r;f : t > 0} on 
Eat. For any 9 ^ {9n : N > 1), properties and (1200]) hold for {r]f" : t > 0} 
if and only if they do so for the speeded up process {?7g^j ■ t > 0}. Furthermore, 
condition (VI) remains invariant by any re-scale of time, while (|5.ip and the as- 
sertion preceding it are implied by the corresponding claims for {rjg^^ : t > 0}. In 
consequence, speeding up the process appropriately, we may assume in Proposition 
[Qthat 

rAr(W,S") = 1 VA^ > 1 (5.2) 
and condition (12.101) becomes 

lim sup E^[r^(W)] = 0. (5.3) 

To prove the last two assertions of Proposition lS.ll we show that the law of Ts^ (W) 
under P^n converges to a mean-one exponential distribution and that 

lim E„«[r3-(W)] = 1. (5.4) 

N—>co 

We identify the trace process {rj^'^ ■ t > 0} with the first marginal of the 
Eat X {0, l}-valued Markov process {{r]^" , X^) : t > 0} defined as follows. To keep 
notation simple, let a; = 1 — x, : E = 0, 1. The transition rates for {(yyf " , ) : i > 0} 
are the following: 

• From each (77, x) € Wn x {0, 1}, the process jumps to x) (resp. to (^, x)) 
with rate R%{r],£_) for any ^ G Wn (resp. for any ^ £ B'^)- 

• From each (?/, x) G x {0,1}, the process jumps to {£,,x) with rate 
R%{v,0, for any C e Eat. 

Let P(,,,a;), (77, x) e Eat X {0, 1}, be the probability measure on £'(IR+, fiw x {0, 1}) 
induced by the Markov process {{rif",X[^) : t > 0} starting from (rj^x). Hence, 
for any starting point {ri,x) G En x {0, 1}, the law of the marginal {rj^ ™ '■ t > 0} 
on -D(M+, £Af) under P(n.x) is Pr;- 

By Proposition 16. 3[ the conditioned probability measure /i^( •) :— iin{ ■ \ En) is 
the invariant probability measure for the trace process {77^" : t > 0}. Define the 
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probability measure on Em x {0, 1} by 

mNiv,x) = (1/2)^^(7?), for (ry,x) e Eat x {0,1} . 

We may check that rriAr is an invariant probabihty measure for {(fyf " , X^^) : t > 0}. 
In particular, {{rj^ : t > 0} is positive recurrent. 

Clearly, for any r/ e Wn, the law of T3c(W) under coincides with the law of 
the first jump 

M{t>0:X^^X,^} 

under 3,), for any x G {0, 1}. Hence, to prove that T3c(W) converges to a mean 
one exponential law it is enough to show that the second coordinate of the trace 
of the process {{rif'^,X[^) : t > 0} on Wn x {0, 1} converges to a Markov process 
on {0, 1} which jumps from x to 1 — x at rate 1. This is done in two steps. We 
first prove in Lemma 15.31 that the sequence of processes {X^ : t > 0} is a tight 
family. Then, we characterize in Lemma 15.61 all limit points by showing that they 
solve a martingale problem. Both statements rely on a replacement result, stated 
in Lemma 15.21 and Lemma 15.51 which allows the substitution of a function by its 
conditional expectation. 

Conditions (|2.9p and (|5.3p imply that 

lim sup eJ / ^ {RNiVs") + 1} 'i-iVs" e WN}ds\ = 0, (5.5) 

where r^(£) = {£.) := inf{i > : rjf" = ^^}. As a consequence of dES]), we get 
the following lemma. 

Lemma 5.2. For every t > 0, 

ft 



lim sup 







= 0. 



Proof. Recall the notation introduced in Subsection 16.21 Let g : £jv — > M be given 
by g{v) — R^M^iv £ Wn} so that the expectation of g with respect to fif^ is 
equal to HN{WN)/f^N{£-N) in view of (|5.2p . Consider the partition tt = {Wn, B'j^} 
of Eat and note that the conditional expectation {g\Tr)^£^ — 1{?7 £ W^jv}- Since g is 
integrable with respect to /x^, the statement follows from (j5.5p and CoroUarv 16.51 
applied to the process " : t >0}. □ 

We use this lemma to show tightness for the sequence {X^ : ^ > 0}. 

Lemma 5.3. Fix an arbitrary point ij — (77^ : > 1) m W and z £ {0, 1}. For 

each N > 1, denote by <Qn the law of {X^ ■ t > 0} under P(^n ^-j. Then the 
sequence of laws {Qn ■ N > 1) is tight. 

Proof. For each T > 0, let It denote the set of all stopping times bounded by T. 
By Aldous criterion (see Theorem 16.10 in [2]) we just need to show that 

limlimsupsup sup P(„n [ jAT^g ~ X^\ > el = (5.6) 

^10 N^oD e<5Te%T 

for every e > and T > 0. Denote by L^ the generator of {r/f" , X[^)t>Q and by 
p : £jv X {0, 1} — > {0,1} the projection on the second coordinate. Consider the 
martingale 



Ml 



N 



Xr ~ ~ f iL%p)i^f -,X^)ds. 
■Jo 
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It is therefore enough to show that (15. 6p holds with X^^g — replaced by 

M?^, - M? andby /;+''(Li.p)(r;f«,Xf)d.. 

Consider the integral term. By Chebychev inequality and by the strong Markov 
property, we need to prove that 



lim lim sup sup 



sup 



E 



N\ 



ds 



0, 



SiQ M^oo e<5 (r,,a;)e£jvx{0,l} 

where E(,j j.) stands for the expectation with respect to .j). A simple computa- 
tion provides 

(Li,p)(ry,a;) = {x - x] R]^ {Ti)l{r^ ^Wn] ■ 
The proof is thus reduced to the claim 



lim lim sup sup / i?]^(ryf ") l{r/f " e M^jvl 

"5X0 jv— oo jjefi '- 



0. 



(5.7) 



Since the expectation above is less than or equal to 



E, 



5, 



the limit (|5.7p follows from Lemma [5.21 

We now turn to the martingale part {M^ '■t>Q}, whose quadratic variation is 
given by 



(M)f 



By Chebychev inequality 



f {L^(p^) - 2pL^p}(r;f",Xf )ds 
Jo 



AT ■ 



Finally, by the explicit formula for the quadratic variation and by the strong Markov 
property, the right hand side above is less than or equal to 



1 



— sup E,, 
It remains to use (|5.7I) 



□ 



As a consequence of Lemma l5.3l we obtain condition (VI) for the triple (W, W, ^) 
with respect to the trace process {r/f " : t > 0}. 

Lemma 5.4. For any point ij — (77^ : > 1) m W, 

lim P^n[T^{W) < T-B^iW)] ^ 1. 

Proof. Fix T]^ in Wn^ N > 1. Consider the modified uniform modulus of continuity 
co's:D{R+,{0,l})^ 



M+ given by 
uj's{x.) :— inf max sup 

{t,} 0<i<r t,<s<t<U+i 



Xt Xs 
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where the first infimum is taken over all partitions {ti : < i < r} oi the interval 
[0, 1] such that 

r = to <ti < ■ ■ ■ <tr = T 

\ ti — > S , for i — 1, . . . , r . 

By the previous lemma (see e.g. Theorem 1.3 in Chapter 4 of [E]), 
limlimsupP(,,jv o)[c^5(X^) = l] =0. 

<5iO AT^+oo 

Therefore, since for all (5 > {T33c(W) < S} C {w^(X^) = 1} P(^iv^)-almost 
surely, 

lim lim inf P„iv [ T^c (W) > (5 1 = 1 . (5.8) 

On the other hand, by (|5.3p . we have 

lim P„iv[Tg(W) >S] = (5.9) 

for any S > 0. The desired result follows from (j5.9|) and (|5.8|) . □ 

Actually, since {T^(W) < Tse(W)} C {T^ < Tse} , Lemma [O] proves condition 
(VI) for the triple (W, S, ^) with respect to the process {?/^; t > 0}: For any point 
T, = (jy^ : iV > 1) in W, 

lim P„iv[Tf < Ts-l = 1- 

We now consider the trace of {{r]f",X^) : t > 0} on Wn x {0, 1}, denoted by 
{{rj^'' ,Xi^") : i > 0}. As we shall see in Section^ since {(r/f",Xf ) : t > 0} is 
positive recurrent, the trace process {{r]^" , X^"^) : t > 0} is positive recurrent as 
well. Moreover, the invariant probability measure for the trace process, denoted by 
, coincides with rriTv conditioned to Wn x {0, 1}: 

m]^(?7,x) := i^/2)^l^{r]), for (r,,x) G W^^v x {0,1}. 

The marginal process {r]^" '■ t > 0} corresponds to the trace of {rj^" : i > 0} on 
Wn. 

Let L"^ denote the Markov generator of {(r/J^",Xf'") : t > 0}. Define, in 
addition, the Markov generator C as 

CF{x) := F{x) ~ F{x) , x e {0,1} , (5.10) 

for every F : {0, 1} R. For each iV > 1, let p = p^r be the projection function 
on the second coordinate p : Wn x {0,1} ^ {0,1}. If R]y(',') stands for the 
transition rates of {(ryj^",^^'") : i > 0}, we have that 

L^{Fop){rj,x) = {F{x)-F{x)} ^ {{v, x), {^,x)) 

for any {ri,x) G Wn x {0,1}. By applying CoroUarv 16.21 to the Markov process 
{{r]t",X[^) ■.t>0} and its trace on Wn x {0, 1}, we get that 

^ R^{ifj,x),{i,£)) = R^iv) 

for all (ri,x) G Wn x {0,1}. Therefore, by (|5.2p . the conditional expectation of 
L]^(F o p), under , given the cr- field generated by the partition 

Wn X {0, 1} = (Wn x {0}) U {Wn x {1}) , (5.11) 
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is {CF) op . Therefore, applying Corollarv l6.11l to the trace process {(jyj^™ , X^") ■ 
t > 0}, the function 'L'^{F o p) and the partition (|5.1ip . we obtain the foUowing 
replacement lemma. 



Lemma 5.5. For every x € {0, 1}, function F : {0, 1} 



and time t > 0, 



lim sup 



E 



{Lj^iFop){r,f'«,Xf'-) - CFiXf-)}ds 



Proof. Recall that the conditional expectation of L]^(F o p) is (CF) o p . Since 

\L^{Fop){rj,x) £F{x)\ < «(77) + 1) max{|F(0)|, |F(1)|} , 

in view of Corollary 16.51 to prove the lemma we just need to check that for any 
{0,1}, 



lim sup E( ) 
where, for each > 1, 



(i?)5'(,7r") + l)l{^f"=2:}ds 



0, 



(5.12) 



T(I.) = T(>^,.)W := inf{i>0:(77r",^r") = (e^,^)}- 
Fix an arbitrary x G {0, 1}. It follows from conditions (|2.9p and ()5.3p that 



lim sup 

To keep notation simple, let us denote 



0. 



(5.13) 



-AT := 



1) " =x}ds. 



Since {T^{W) < Tse(W)} C {T^(W) = T^,^)}, by LemmaEH (O^ and Cheby- 
chev inequality, for every t > 0, 

lim sup P(^,.)[Tjv > t] = 0. (5.14) 

By the strong Markov property, (|5.14p and the arguments presented in the proof 
of Lemma 14. 3[ 

lim limsup sup x)[Tn 1{Tn > A}] = 0. 

A^oo AT^oo rieWN 



(5.15) 



Hence, by Lemma 15.4 



limsup sup E(,;,,)[Tjvl{r^(W) >rsc(W)}] = 0. 



On the other hand, 

E(„^,)[Tjvl{r^(W) <T3c(W)}] < E„ 



□ 



Therefore, (|5.12p follows from this estimate, (|5.13p and (|5.15p . 

We now prove the convergence in law of {X^'^ : t > 0} as N oo. Fix an 
arbitrary point t] — (77^ : iV > 1) in W. For each iV > 1, denote by Fn the law of 
{Xf'" : i > 0} under P( 

^jv^o)- Following the same argument presented in the proof 
of Lemma [5.31 we can show that (Pjv : > 1) is tight. 
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The uniqueness of limit points for this sequence is established as follows. Assume 
without loss of generality that Pjv ^ P, as ^ oo, for some probability measure P 
on D{R+, {0, 1}). For t > 0, let Xt denote the time-projection Xt : D{M.+ , {0, 1}) 
{0, 1}. We shall prove in the following lemma that P solves the martingale problem 
associated to the generator C defined in (|5.10p . It is well known that this property 
together with the distribution of Xq, characterize the measure P. 

Lemma 5.6. Under P, Xq = a.s. and 



Mf ^ 



F{Xt) - FiXo) - f CF{Xs) ds, forty Q, 
Jo 

is a martingale for any function F : {0, 1} —>■ R. 

Proof. The first claim is trivial. For the last one, fix < s < i, a function F : 
{0,1} E and a bounded function U : D(R+,{0,1}) i-^ M depending only on 
{Xr : < r < s} and continuous for the Skorohod topology. Denote by E and En 
the expectation with respect to P and P^r, respectively. We shall prove that 



E [m[u] = E [Mfu] 



(5.16) 



Recah that L]^ denotes the generator of {{rjf''' , X^'') : t > 0}. For A^ > 1, 
consider the P(j,]v o) -martingale {AI^^ : t > 0}, defined by 



N 



= F{X^- )-F{0)- f\^{Fop){ijY-,X^-)ds., 
Jo 



t>0. 



Denote C/^ := C/(A.^"). As {M^^ : t > 0} is a martingale. 



so that 



E 



(r,",0) 



E(,«o)[Mff/^] ^E(,«o)[MfC/^] 



{F(Af'" ) - F(Af « )- j\^{Fo p) (Tyf " , X^" ) dr } 



= 0. 



On the other hand, since is bounded and JT^ -measurable, it follows from the 
Markov property and Lemma 15.51 that 



lim E 



(r,",0) 



J' [-Lj^iFo p) (77,^" , X^" ) - CF{X^" ) } dr 



= 0. 



Putting the last two assertions together we get 

\u\ 



lim ] 

N^oo 



^[F{Xt) - F{Xs) - ^ CF{Xr) dr] 



= 0. 



(5.17) 



Now, since Pjv converges to P, time averages of EAr[F(Xf)C/] and Eisi\F{Xs)U'\ 
converge to time averages of E [i^(At)t/] and E [F(As)C/], respectively. Hence, 
from this last observation and (|5.17p it follows that 



dr\ 



U 



{F{Xt+r) 



Fix, 



s+r) 



f+r 



CF(Xs)} 



= 



for every e > 0. It remains to let e | and use the right continuity of the process 
to deduce (|5.16p . which concludes the proof of the lemma. □ 
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Under P, {Xt : t > 0} is therefore a Markov chain on {0, 1} with generator C 
and starting at 0. We have thus shown that, the law of 

Ta3c(W) inf {i > : X^" ^ 1} , 

under P^^n ^), converges to a mean-one exponential distribution. To conclude the 
proof of Proposition 15.11 it remains to check (|5.4p . By Lemma 15.41 and the conver- 
gence in law of T'^^i'W), (W, W,^) is a valley for the trace process {jy^ ™ t > 0} 
with depth 1. Therefore, applying item (m) of Proposition [13] to ('W,W, ^) and 
{Vt" ■ t > 0} we get (|5.4|) . This concludes the proof of Proposition [5lT] □ 
We are now in a position to prove Theorem 12.71 Condition (VI) follows from 
Proposition 1 5 . 1 1 and Condition (V3) from (|2.11[) and Chebychev inequality. Con- 
dition (V2) follows from (V3) and the convergence in law of ryvCW, S'^) T^c (W) 
stated in Proposition [O] 

Proof of Theorem 12.81 Next result is the main step in the proof of Theorem l2.8l 



Proposition 5.7. Recall the notation introduced in Subsection \2.2\ If conditions 
(C2), (C3) and (HO) are in force, then so are (Ml) and (M2). 

The proof of this result is divided in three lemmas. As in the proof of Proposition 
15.11 without loss of generality, we may assume that 9n = I, ViV > 1. In this way, 
condition (HO) guarantees that, for every x,y G S, x ^ y, 

lim rAr(£-,£^) = r{x,y), (5.18) 

and we shall prove the convergence in law of the sequence {X^ : t > 0}, iV > 0. 
Clearly, conditions (C2) and (C3) imply 

lim sup E„ / {i^^(ryf«)+r^(£^£")}l{r;f" e£^}ds =0 (5.19) 

for any x £ S, where {rjf" ■ t > 0} stands for the trace process of {iit : < > 0} on 
£-N. Let us define V^v ■ £jv > K as 

xes 

Let /i^ be the measure /ijy conditioned to £jv and denote by the /i^-conditional 
expectation of Vn given the cr-algebra generated by the partition E^n = ^xeS^N- 

Vn{v) ■■= ^ ^n} , e £a . 

Since Vat is integrable with respect to /i^, it follows from CoroUarv 16.51 and from 
that, for any t > 0, 



lim sup 



{VN-VN]{Vs'')ds = 0. (5.20) 



In order to prove (M2), fix some x £ S and a point rf ~ (jj : > 1) in £^. For 
each A" > 1, denote by Pat the law of {Xf : t > 0} under P^n. The convergence of 
the sequence {Vn ■ N > 1) stated in (M2), follows from tightness and uniqueness 
of limit points. We first examine the tightness. 

Lemma 5.8. The sequence (Pat : > 1) zs tight. 
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Proof. For each T > 0, let Tt denote the set of aU stopping times bounded by T. 
By Aldous criterion (see Theorem 16.10 in [2]) we just need to show that 



lim lim sup sup [ \X^_^g -X^\>e] = 

<5iO N^co g<5 ^g<j^ 



(5.21) 



for every e > and T > 0. 

Let be the generator of the trace process {rjf" ■ t > 0} and let {M^^ : ^ > 0} 
be the martingale defined by 



N 



X 



N 



X, 



N 



To prove tightness, it is therefore enough to show that (|5.2ip holds with the differ- 
ence X^^g - X^ replaced by - and by /^^^ L%Xfds. 

Consider the integral term. By Chebychev inequality and by the strong Markov 
property, we need to prove that 



lim lim sup sup 



An elementary computation shows that 



= 0. 



x,y£S 

for any 77 G Sat. Since |L^^'jv| < kVn, the proof is reduced to the claim 



lim lim sup E„ 

SIO Af^oo ^g£^ 



0. 



The left hand side can be written as 
-'v 



lim lim sup <. E, 

SIO N^oo ^g£„ I ' 







{Vn -VN}{Vs")ds 



E, 



The first term converges to zero as t for any 6 > 0, by (|5.20|) . The second 
term is bounded above by 

lim lim S r7v(£^,£^), 



xeS 



which is equal to zero by (|5.18p . 

We now turn to the martingale part, whose quadratic variation, denoted by 
(M^)t, is given by 



(M 



,rN\ 



{L%{^Ny{ri!n - 2Xf{L%^M){Vs'')} ds, t>0. 



An elementary computation shows that this expression is equal to 
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By the explicit formula for the quadratic variation, by Chebychev inequality and 
by the strong Markov property, 



< 



sup 







It remains to repeat the arguments presented for the integral term of the decom- 



position. 



□ 



Now we turn to the uniqueness of limit points. Assume without loss of generality 
that the sequence Qat converges to a measure P . Denote by and £ the Markov 
generators on the state space S — {1, . . . , k} given by 

(£jvF)(x) = ^ {F(2;)™F(x)}r^.(£^£^) 

yi^S\{x} 

and 

(£F)(x) = {F{v)-F[x)}r{x,y) . 

y(iS\{x} 

For t > 0, let Xt denote the projection D{R+,S) ^ S. The probability P is 
completely determined by the properties stated in the following lemma. 



Lemma 5.9. Under P , Xq = x and 



Mt = F{Xt) - F{Xo) - f £F(X,)ds 

JQ 



(5.22) 



is a martingale for any function F : S i 



The proof of this lemma follows closely the one of Lemma [5T6l It suffices, in par- 
ticular, to show the following replacement lemma. Let stand for the generator 
of {yyf « : t > 0}. 

Lemma 5.10. For any t > 0, 



N 



lim sup 



{L%iFo^^JN)-iilF)o■qJN}iVs")ds\ =0 

Proof. First, by condition (HO), we have that 

lim sup E J / {(£jvF)(Xf ) - (£F)(Xf)}dsl = 0. 

It remains to prove that 

lim sup E J / {L%{Fo^SJN)-iilNF)o-qjN}iVs")ds\ ^ 0. 

The ^^-conditional expectation of L^{F o v]/^) given the cr-algebra generated by 
the partition fi^r = [J^^s^^n (£Ari^) o^'at- The expectation of | L%{F o'^ jq) \ with 
respect to ^% is bounded by C[F) X^ajes ''"n{£-^ , for some finite constant C{F), 
depending only on F, and, for any 77 G Eat, \Lfq{F o 'iiN){r]) — (£jv^') ° ^'a'('7)| is 
bounded above by 

2 max |^^(^)| J2 Wv) " r•^(£^ Hv e £^} . 



(5.23) 
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By Corollary EH applied to g ^ L%{F o ^ n) and by ^JQij, ^5^2^ holds, which 
concludes the proof of the lemma. □ 

This concludes the proof of condition (M2). Condition (Ml) follows from Propo- 
sition [STT] with W = 23 = £^ U A, which concludes the proof of Proposition 
O □ 

For Theorem 12.81 it remains to check condition (M3) for non-absorbing states. 
This follows from Proposition [23] since condition (M2) has already been deduced. 

Proof of Theorem l2.9l We assume in this subsection that the process is reversible 
and adopt all notation introduced in Section [2l The proof of Theorem 12.91 relies on 
the following result which states the important fact that, under condition (|2.15[) . the 
capacity between W and CB"^ is asymptotically equivalent to the capacity between 
any point of W and 

Proposition 5.11. Consider two sequences of sets W and 23 satisfying (|2.3p . 
Assume that condition (2.15\) holds for some point $ = : N > 1) in W. 

Then, the assertions of Proposition \5.1\ are in force. Moreover, for every point 
iC'^ : N >1) in W, 

lim '-^^^4^ = 1 , (5.24) 
iv^~ cap^(C,S^) ^ ' 

and 

lim inf PJTc<T^c] = 1. (5.25) 



Proof. We have shown just before the statement of Theorem 12.91 that conditions 
(|2.9[) . (|2.10[) follow from (|2.15p . In particular, the assertions of Proposition lSTl hold. 

Fix an arbitrary point C = (C^ : > 1) in W. By (|6.16p applied to 
g = 1{W}, {C^}, and to {C^}, g = 1{W}, {C^}, for any 77 = (ry^ : iV > 1) in W, 



E^iv [ (W) ] < E„iv [ ( W) ] E^iv [ ( W) 

Mjv(W) 

capAr(J7,' 
2AiJv(W) 



- cap^(,7,0 ^ + cap^(C,0 ^ ^ ^ 



< 

capjv(l) 

From this estimate, identity ()2.14p and hypothesis ()2.15p . it follows that 
lim rAr(W,S")E„iv[TjW)] = 0, 

which, by (|5.ip in Proposition [5?T1 iniphes that 



A^^oo E,.[T^.(W)] = 



This limit corresponds to item (i) of Proposition 14.41 with the point C, instead of ^. 
Item (ii) of Proposition 14.41 follows from the last two assertions of Proposition l5.ll 
From items (i) and (ii) we conclude that (W, W, (^) is a valley for the trace process 
{ryf " : t > 0}. Hence, 

hm p^«[rc(£) <r3c(£)] = 1, 

which implies condition (VI) for the triple (W, C) because {r^(£) < T3c(£)} C 
{Tc < Tsc}, proving ({5^5)) . 
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By Proposition [QOl with A = {77}, B ^ T," and g = 1{W}, and by identity 
p.l4p . the limit ()5.ip can be re-written as 



lim 



{1{Wn}, fNiv, 23^))^„ cap^(W, S^) 



AiAr(W)capjv(r7,®'=) 

Replace t; by C in this formula. By (|5.25p . the infimum of /Ar(C,6^) over Wn 
converges to 1 as iV f c». Therefore, (|5.24p follows from this observation and the 
previous identity. □ 



We are now in a position to prove Theorem l2.9l We first show that (W, 23, ^) is 
a valley of depth 0n = rArCW,^^)-! = ^Ar(W)/capjv(W, S^). Identity (|2l4l) and 
Proposition 16 . 1 01 show that 



_ l{Ajv},/Ar(r7,a3^))^„ cap^(W,a3^) 
^"'^■^'"^-'^ M^(W)cap^(r,,S^) 



r^(W,S^)E„«[T3c(A)] = 



By Proposition lS.llI (|5.24p holds. Since fNiv,'^") is bounded by one, (I5.24p along 
with hypothesis ([^1^ proves ^J^. Since and ([^1^ follow from ^J^, all 
the hypotheses of Theorem l2.7l are fulfilled. Therefore, (W, 23, ^) is a valley of depth 
On = rAr(W, 'B'=)-i = /i7v(W)/capjv(W, 3^=). Last identity follows from Lemma[6Jl 
Fix now a point ^ in W. To prove that (W, 25,^) is a valley, we check condi- 
tions (i)-(iii) of Proposition 14.41 Property (i) has been proved in (|5.26p . Since 
(W, 23,^) is a valley, conditions (ii) and (iii) are in force due to the first part of 
Proposition Hence, by the second part of this proposition, (W, 23, ^) is a valley 
of depth E^N [Tsc (W)]. Finally, since (W, 23,^) is a valley, by the first part of this 
proposition. On and E^]v[r3c(W)] are asymptotically equivalent sequences. 



Proof of Theorem 12.101 We need to check that all assumptions of Theorem [2 
are satisfied. As in the proof of Theorem 12.91 conditions (C2), (C3) follow from 
assumption (HI). It remains to show that (CI) is fulfilled for all non-absorbing 
states. Fix such a state x. It is enough to prove that 

lim sup sup -5-E,Jr£,(A) 1 = 0. (5.27) 

By Proposition [6TO] and since /^gx is bounded by 1, the expectation is less than or 
equal to /^7v(A)/cap(77, £^). By (I5.24p . we may replace asymptotically r/ by in 
the previous capacity. By Lemma [^?71 cap(£^,£^) is equal to ij.n{^^)tn{^^ ,^^)- 
In conclusion, we have shown that 



lim sup sup -^'Ein 



Tp,{A) < lim sup i ^^^^4^- (5.28) 



Since a; is a non-absorbing point, by assumptions (HO), (H2), the right hand side 
is equal to 0. This concludes the proof. 

Proof of Remark 12.111 We need to show that (H2) holds for non-absorbing 
states and that (M3) holds for absorbing states. Clearly, (H2) follows from (H2') 
for non-absorbing states. On the other hand, by Proposition 15. 71 (M2) is fulfilled. 
Hence, by Lemma [4.71 (M3) for absorbing (and non-absorbing) states is a conse- 
quence of (|5.27p . By (|5.28p . assumption (H2') implies (I5.27p . which concludes the 
proof. 
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6. Continuous time Markov chains 

We state in this section several properties of continuous time Markov chains 
used throughout the article. We start assuming that the holding rates are strictly 
positive and finite and that the jump chain associated is irreducible and recurrent. 
We add assumptions as we progress. At the end, we consider the case of positive 
recurrent, reversible Markov chains whose holding times belong to L^{fi), where fi 
is the unique invariant probability measure. 

Consider a countable set E and a matrix R : E x E ^ M. such that R{ri, ^) > 0, 
7j ^ < Riv.v) < 0, = 0, 77 e Let A(r7) - -R{ri,r,). 

Since X{r]) is finite and strictly positive, we may define the transition probabilities 
{P{V,0 ■V,^^^ E} as 

and p{r],r]) = for -q € E. We assume throughout this section that {p{r],£,) : 
?7, ^ € E} are the transition probabilities of a irreducible and recurrent discrete 
time Markov chain. 

We claim that there exists a unique stochastic semigroup {pt : t > 0} on E 
satisfying 

,^P*M)_^£M) = and MV,0 - S,, (6.2) 

for every rj, £^ € E, where is the delta of Kroenecker. To prove the existence, 
we construct a Markov process {rjt : t > 0} on E whose Markov semigroup satisfies 
(16. 2[) . We shall use this construction in some of the proofs below. 

Let Y — {Yn : n > 0} be an irreducible, recurrent, £^-valued discrete time 
Markov chain with transition probabilities {p(77,0 ■ Vi^ ^ E} given by (16. 
Let (e„ : 71 > 0) be a sequence of i.i.d. mean one exponential random variables, 
independent of Y. We associate to every sample path of Y the sequence of random 
times T = (r„ : n > 0) given by 

" " A(r„) • 

Since Y is recurrent, ^i^Q^i = oo a.s. In particular, the time-change 

n 

a{t) = min{?7^ > : > i} (6.3) 

1=0 

is a.s. finite for every t > and rjt — ^0(4) is a-S. well defined for all i > 0. In 
Theorem 2.8.1 of [T7] it is proved that {774 : t > 0} is a strong Markov process with 
respect to the filtration {J^t : i > 0}, JFt = a{rjs : s < t). The stochastic semigroup 
corresponding to {774 t > 0} fulfills (|6.2p . as follows from the proof of Theorem 
2.8.4 in [T7]. On the other hand, the uniqueness of the stochastic semigroup is a 
consequence of Theorem (51) in Chapter 7 of [TT] along with the recurrence of the 
transition probabilities p(-, •). Note that there is no explosion since X]i>o -^i ^ ^ 
a.s. 

In conclusion, a collection of nonnegative numbers {i?(77, ^) '■ i],^ (z E} satisfying 
the conditions listed at the beginning of this section determines uniquely the law of 
a strong Markov process {774 : i > 0}. We shall refer to R{-, •), A(-) and p{-, •) as the 
transition rates, holding rates and jump probabilities of {rjt t > 0}, respectively. 
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The Markov chain Y — {¥„ : n > 0} is called the jump chain associated to {rjt : 
t > 0}. 

Of course, since the jump chain Y is irreducible and recurrent, so is the corre- 
sponding Markov process {rjt '■ t > 0}. In consequence, {rjt '■ t > 0} has an invariant 
measure which is unique up to scalar multiples. Moreover, 

M{^) X{v)Kv) , V^E, (6.4) 

is the invariant measure for the jump chain Y , also unique up to scalar multiples. 
The proofs of these assertions can be found in Sections 3.4 and 3.5 of [T7]. 

Recall that ta ■ D(R+,E) R+, A C E, denotes the hitting time of the set A: 

TA{e.) = M{t > : et e A} . 

Let Ta '■= TA{r].) and :— -q <E E. Define the stopping time t\ : Z?(K+, E) 
as the first return to A: 

T^(e.) inf{t > : Ef e A, 7^ Co for some < s < t} , 

and let T+ T+(r,.), T+ T+j, e E. 

Let P,,, ?7 e i?, be the probability measure under which the jump chain : 
n > 0} and the Markov chain {rjt : t > 0} start from ry. Expectation with respect 
to P,, is denoted by E,,. It follows from the proof of Theorem 3.5.3 in [IT] that for 
any rj & E 

MO - E„ / " l{rj,=Ods , CeE, (6.5) 
L Jo 

is an invariant measure for {r]t : t > 0}. 

6.1. The trace process. We present in this subsection some elementary properties 
of trace processes and we state some identities used throughout the article. 
Let h : E ^ K+ be a nonnegative function with nonempty support F: 

F := {t] e E : h{T]) > 0} ^ . (6.6) 

Define the additive functional {T^'' : t > 0} as 



r/' := fh{ru)d3 
Jo 



Notice that T/" e R+, P^-a.s. for every t] e E and t > 0. Denote by {Sl" : t > 0} 
the generalized inverse of 7^'' : 

Si' := sup{s > ; Tj' < t} . 

Since {r/t : i > 0} is irreducible and recurrent, limt^oo ~ P,,-a.s. for every 
1] £ E. Therefore, the random path {t]^ : t > 0}, given by t]^ = rjgh, is P^-a.s. well 
defined for all 77 G and takes value in the set F. We call the process {77^ ■ t > 0} 
the /i-trace of {r/t : t > 0}. Clearly, {rj^ t > 0} coincides with the trace of 
{Vt-t> 0} on F, defined in Section[l if h ^ 1{F}. 

A change of variables shows that for any subset B oi F and for any function 
/:F^R+, 

rTsiv!") i-Tb 

f{rj';)dt = / f{vt)h{r^t)dt (6.7) 
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Pr,-a.s. for every 77 e E. This identity also holds if we replace TB{ri'^), Tb by 
Tg{rj^), Tg , respectively. Furthermore, for any two disjoint subsets A, B of F, it 
follows from the construction of the Markov chain : t > 0} that 

P„ < TB(r;^) ] = V,,[Ta < Tb ] 

for all ry in F. This identity needs to be reformulated if we replace the hitting times 
by return times. Indeed, if the process starting from ry returns to F by ry, while in 
the original version the process returned to ry, in the trace version the process never 
left rj. We claim that for all ry G _F and all disjoint subsets A, B of F, 

Pr,K(^') <^b(^'')] = ^v[t1<T+\T+ = Tf\{^}] . (6.8) 

To derive this identity, intersect the event {r^irj'^) < T^{ri'^)} with the partition 
{Tp = rp\{^}}, {Tp = r+} and apply the strong Markov property to the second 
piece to get that 

To conclude, observe that on the set {Tp = Tp\_^,^j} we may replace 7y^ by rj. in 
the event {r^iri^) < (??.'')}• 



Proposition 6.1. Under {P^ : rj G F}, {rj^ : t > 0} is an irreducible, recurrent, 
strong Markov chain with transition rates given by 



R^'iv^O = ^^Pv[T+ = T+], rj,^eF,r,^^. 



Proof. Recall the explicit construction of the Markov chain {rjt : t > 0} presented in 
the previous subsection. To derive the /i-trace from this construction, we consider 
first the trace of the jump chain {Yn : n > 0} on i^. 

Define the sequence of times {t„ : n > 0} as to = 0, ii = inf{n > I Yn E F} 
and tn-f-i — tn + ti o Qtn, n > 1, where {Qk ■ k > 1} are the discrete time shift 
operators. Let F'' = {Y^^ : 71 > 0} be given by Y^ — Yt^ . When the jump chain 
{Yn : n > 0} starts in F, Y^ = {Y^ : n > 0} defines a F-valued discrete time 
Markov chain with transition probabilities 

p{v,0 = F^[T+=T+], v,^eF. 

Note that p{ri,ri) may be strictly positive and that Y'^ inherits the irreducibility 
and the recurrence properties from Y . 
Let T") = {T^ : n > 0} be the sequence 

T^, = h{Yn)-^\- , ri>0. 

By definition, the ft,-trace of {r/t : t > 0} is given by r/^ = ^Q(t)' ^ — where a(-) 
represents the time-change (16.31) with Y'^ and T'' in place of Y and T, respectively. 
Note that {et„ : n > 0} is a sequence of i.i.d. mean one exponential random 
variables independent of the process {Y^ : n > 0}. By this observation and by the 
proof of Theorem 2.8.1 in {r]'^ : t > 0} is a strong Markov process on F. 
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The irreducibility and the recurrence of {77^ ■ t > 0} are inherited from the 
process F''. On the other hand, the transition rates ■ Vi^ ^ P} of 

strong Markov process {rj^ : t > 0} are given by 

for ri,£^ ^ F, r] ^. The second identity foUows from the proof of Theorem 2.8.4 in 

m- □ 

It foUows from this proposition that the holding rates {A'''(ry) : r/ G F} and the 
jump probabihties {^''(77, £,) ri,£^ £ F} of the /i-trace process {r]^ t > 0} are given 

by 

^'(^^ = ^^"^^^^^^^{.1]' (6.9) 

and, for rj ^ ^, 

Note that p^{-,-) depends on h only through its support. The second identity is 
obtained by intersecting the event {Tf^^i^j = T^} with the partition {Tp = Tp\|-^|}, 
{Tp = r,^} and applying the strong Markov property to the second piece as in the 
proof of 



When h is the indicator function of a set F , we obtain an explicit formula for 
the transition rates of the trace process. 

Corollary 6.2. Let stand for the transition rates of {77^ '■ t > 0} when h = 
\{F}. Then, for rj, ^ in F, rj ^ ^, 



Proof. By Proposition 16.11 i?^(?7,^) = X{r])Prj[Tp = T^]. Consider the stopping 

I pc WlLii LliC i_:UiiVL:iitiUil LlitlL -L pc 



time Tpa with the convention that Tp^ = 00 if F"^ = 0. Decomposing the event 



{Tp = T^} according to the event {Tp < Tpa} and its complement, we get 

R^iv, - Hv) [ T+ - T+ ; T+ < r+ ] + A(77) P„ [ T+ = T+ ;T+<T+]. 

The first probability on the right hand side is equal to P^lTp ~ T^] — piViO^ 
while the second term, by the strong Markov property, is equal to 

J2 RivX)Pc[TF = T^]. 

This concludes the proof of the corollary. □ 

The previous corollary provides an explicit formula for the rates R^ in terms of 
the holding times A and the transition probabilities p in the case where F = E\{£^q}: 

R^{v,0 = R{v,0 + R{v,^o)p{Co,0 

for r] ^ ^, {77,$} E\ {(,()}. In particular, if _E is a finite set, the rates R^ can be 
obtained recursively. 
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Since {77^ : t > 0} is recurrent and irreducible, it has an invariant measure which 
is unique up to scalar multiplies. Let fj, be an invariant measure for {rjt '■ t > 0} 
and denote by /ijj the measure on F given by 

Proposition 6.3. /i^ is an invariant measure for {rj^ : t > 0}. In particular, if h 
is jjL— integrahle then {77^ : t>Q} is positive recurrent. Moreover, if ji is a reversible 
measure for {rjt : i > 0} then /i^ is a reversible measure for {ri^ : i > 0}. 

Proof. Without loss of generality, we may suppose that /i is of the form (|6.5p for 
some r] e F. Thus, by (|6.7p . for any ^ & E, 

'-Jo Jo 

This shows that /i^ is an invariant measure for the h-trace process. The second 
assertion follows from Theorem 3.5.3 in |17j . 

Suppose now that /i is reversible for R{-,-). Then, the measure M defined in 
(|6.4p is a reversible measure for the jump chain Y = {¥„ : n > 0}. Since the events 
{T+ = T+} and {T+ = T+} depend only on Y, 

M{rj) P, [T+=T+] = M{0 [ T+ = T+ ] , 

for any ri,S^ £ F, rj ^ ^. In consequence, by the formula for obtained in 

Proposition 16. 11 fi'^ is reversible for the /i-trace process. □ 

6.2. Positive recurrent case. We assume from now on that the Markov chain 
{rjt : t > 0} is positive recurrent. Denote by fi its unique invariant probability 
measure. 



Replacement Lemma. For any probability measure v on E, we denote by 
expected value with respect to v. 



the 



Lemma 6.4. Fix a function g : E R with nonempty support, integrable with 
respect to fi and such that {g) ^ = 0. Fix also some in A — {rj : g{'tf) ^ 0}. For 
every t > Q, 



sup 

J)G-B 



g{vs) ds 



< 2 sup E,, 



\g{r]s)\ds 



Proof. Let {0( : t > 0} stand for the time shift operators on Z)(R+, E). Define the 
random times Hq = 0, Hi — and = Hj + o Qfj. (rj.), j > 1. Fix an 

arbitrary r] £ E and let h : E ^ be a nonnegative function, integrable with 
respect to fi. By Proposition l6.31 the trace process {77^ : t > 0} is positive recurrent 
so that 



E„ 



h{f]s)ds ^ E^[t^(77'')] < 00. 



(6.10) 
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Write 



h{r]s)ds 



V E J / hi-qs) ds \{Hj < t < Hj+i} 



h{ris)ds l{Hj < t < Hj+i} 



.11) 



h{r]s)ds l{Hj < t < Hj+i} 



In the last equation, we used the fact that both terms on the right hand side are 
finite. To prove it, notice first that the second term is bounded by the first one. 
By ToneUi's theorem, the first term is equal to 



j>0 k=0 



' =0 

Ee.[ 



h{T]s)ds l{Hj < t < Hj+i} 



fc>0 



h{Tjs)ds l{Hk < t} 



Taking conditional expectation with respect to J-'Ht. , by the strong Markov property, 
this sum is equal to 



E, 



Hi 



h{ris) ds 



Hi 



k>l 



The first term of this sum is finite by (|6.10p . In the second expectation, ^ appears 
instead of rj, and the expectation is equal to (h)^ by (j6.5p . Finally, the sum is finite 
by the strong Markov property and because P^[r^'^ < t] is strictly smaller than 1. 

To estimate the last term in (I6.1ip . note that the event {Hj < t < -f/j+i} belongs 
to J-t and that on this set Hj+i = t+T^oQf[ri.). Therefore, by the Markov property. 



Ee.[ 



j>0 



H, 



h{ris)ds l{Hj < t < Hj+i} 



E, 



E, 



Hi 



h{ris) ds 



Putting together the previous identities, we get that the left hand side of (|6.1ip 
is equal to 



E. 



Hi 



h{r]s)ds] + {h)f,J2P„[Hk<t] - E^ 



fc>i 



E, 



Hi 



h{ris) ds 



Applying the previous identity to and g , since (g)^ = 0, we obtain that 



E. 



givs) ds 



Hi 



9iVs)ds 



E. 



Hi 



9iVs)ds 



We claim that we may replace the stopping time Hi by in both terms of the 
right hand side. Indeed, if rj is different from ^, Hi ~ P,,-a.s. Conversely, if the 
starting point i] is equal to ^, = so that, by (|6.5p . 

f-Hi , . rT, 



Er. 



g{ris)ds 



= E, 



g{ris) ds 
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Thus, taking the supremum over E, we have proved that 



sup 



< 2 sup 



girls) ds 



Finally, since g vanishes outside A and since ^ belongs to A, by the strong Markov 
property, 



Therefore, 



giVs)ds 



E„ 



g{vs)ds 



Ta 



E. 



E 



givs) ds 



sup 



E, 



g{vs)ds 



< sup 



Er, 



giVs) ds 



This concludes the proof of the lemma. 



□ 



Let 5 be a finite set and let tt = {A^ : a: e S*} be a partition of E. Denote by 
/x^, X G S, the stationary measure fi conditioned on A^: /i^(') = Also, for 

each /i-integrable function g denote by {g\TT)^ : — > M the conditional expectation 
of (7, under /i, given the cr-algebra generated by tt: 

The next result shows that if the process thermalizes quickly in each set of the 
partition, we may replace time averages of a bounded function by time averages of 
the conditional expectation. This statement plays a key role in our investigation of 
metastability. It assumes the existence of an attractor, but similar versions should 
exist under weaker assumptions on thermalization. 

For each x G S and /i-integrable function g : i? — > M, let 

5^ := (.g-(5>M»)lMn 

and fix some state £,x in A^ , for each x in S. Next statement follows from Lemma 
16.41 applied to each g^, x G S. Note that the right hand side does not depend on 
time. 



Corollary 6.5. Let g : E 



sup 



E. 



be an integrable function. Then, 







[g - {g\'^)i^]{'ns)ds < 2^ sup e,, 



xes 



\g''{Vs)\ds 







Clearly, the right hand side in the previous corollary is bounded above by 
4^||g|U sup eJ f l{ri,GA-}ds 



where \\g\\a; = sup{|g(?7)l : r/ G A^}. 
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Mean set rates. Let h : E —t M+ be a nonnegative function satisfying (j6.6|) and 
belonging to By Propositions 16.11 and 16.31 {Vt : i > 0} is irreducible and 

positive recurrent. Moreover, its invariant probability measure, denoted by /i'', is 
given by 

- ^A^(^)' ^e^- (6-12) 

For each pair A, B of disjoint subsets of F, denote by rh{A^ B) the average rate at 
which the /i-trace process jumps from A to B: 

where M has been introduced in (|6.4p . We used relation (|6.12p and ProDOsition l6.1l 
in the last equality. We shall refer to r/,,(-, •) as the mean set rates associated to the 
trace process. 

When h is the indicator function of a set _F, we denote r/j by r^?. In this case, 
^x{A)rp{A,B) = ^Af(r;)P4r+<r+^^] . (6.13) 

r^eA 

6.3. The reversible case. From now on, we shall assume in addition that the 
process is reversible with respect to the invariant probability measure /i and that 
the measure M is finite: 

^ M{f^) = Mv)Kv) < oo . (6.14) 

In particular, the mean set rates r^iA, B) are finite. 

Assumption (|6.14p reduces the potential theory of continuous time Markov chains 
to the potential theory of discrete time Markov chains. Recall from Subsection 12.41 
that (■, ■)a/ represents the scalar product in L^(M), that P : L^{M) L'^{M) 
stands for the bounded operator defined by {Pf ){ri) — J^^eEPiv^OfiO^ that 
D(/) = ((/ - P)/, / G L^{M), is the Dirichlet form associated to the Markov 
process {r/t : t >0}. A simple computation shows that for every / in L'^{M), 

DiJ) = ^ E M{^)p{yi,i){f{i)~f{r^)Y. 

Fix two disjoint subsets A, B of E and recaU that C{A,B) := {/ e L'^{M) : 
f{ri) = 1 V ?7 e A and /(^) = V ^ e B}, and that the capacity of A, B is defined 

by 

cap(A, B) := inf { D{f) : / G C{A, B) ] . 

As max{D(/ A l),D{f V 0)} < D{f), V/ G L'^{M), we may restrict the infimum to 
functions bounded below by and bounded above by 1. 
Denote by Jab : E ^ M. the function in C{A, B) defined as 



fAB{ri) P4Ta<Tb] 
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An elementary computation shows that /ab solves the equation 

{Lf)ir,) = r,eE\{AUB), 

fiv) = 1 V^A, (6.15) 
fiv) =0 rjeB. 

Clearly, we may replace the generator L by the operator I—P in the above equation. 
It is not difficult to show that (|6.15p has a unique solution in _L^(M) given by /ab- 
Indeed, if /, g are solutions, D{f - g) = {{I - P){f - g), (/ - g))M = 0. In 
particular, by the explicit expression of the Dirichlet form, f — g is constant. Since 
the difference vanishes on AU B, f — g. 

Lemma 6.6. For any two disjoint subsets A, B of E, 

cap(AS) = DifAB) = Y.M{r,)P,,[T+ <T+]. 

Proof. We first claim that there exists a function / in C{A, B) whose Dirichlet 
form is equal to the capacity cap(>l, _B). Indeed, we have already seen that we 
may restrict the variational problem defining the capacity to functions bounded 
below by and bounded above by 1. Consider a sequence {/„ : n > 1} in C{A, B) 
such that < /„ < 1, lim„^oo -D(/„) = cap(^, S). Since the sequence /„ is 
uniformly bounded, there exist / in C{A,B), < / < 1, and a subsequence, still 
denoted by {/„ : n > 1}, such that f{ri) — lim„^oo fniv) for every rj in E. By 
Fatou's lemma, D{f) < liminf„^oo F){fn) = cap(A, B). Since / belongs to C{A, B), 
D{f ) — cap(A, B), which proves the claim. 

We further claim that / solves (|6.15l) . Fix r] ^ AiJ B. Since / solves the 
variational problem for the capacity, it is clear that f(rf) is the argument which 
minimizes the convex function _F : R — > R defined by 

F{a) = ^M(r^)p(??,0{/(0-a}'- 

In this formula S, rj means that the underlying jump chain may jump from rj to 
^, i.e., that p{ri,^) > 0. An elementary computation shows that the minimum is 
attained at a = J2^Pi''hOf{0 so that f{r]) = {Pf){ri). Since fAB is the unique 
solution in L'^{M) of (|6.15p . / — fAB and cap(A, i?) = D{fAB)- This proves 
the first statement of the lemma. The second one follows from a straightforward 
computation. □ 

In particular, by (j6.13p we have the following very useful identity between ca- 
pacities and mean set rates. 

Lemma 6.7. Assume that F ^ A[J B and AnB = 0. Then, 

fi{A)rF{A,B) = cap(A,B) . 
Next result shows that the mean set rates can be expressed in terms of capacities. 
Lemma 6.8. Let A, B be subsets of F such that An B = 0. Then, 

H{A) rpiA, B) = i|cap(A, F\A)+ cap(S, F\B)- cap(A U S, F \ [A U B]) | . 

Proof. The proof is elementary and follows from Lemma 16.71 and the identity 
2^i{A)rF{A,B) = ^i{A)rF{A,F\A) + fi{B)rFiB,F\B) 
- fi{A U B) rFiA UB,F\[A[JB]). 
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□ 

By assumption ()6.14p . the holding rates X : E ^ M+ belong to This 
property extends to the holding rates {A'' (77) : 77 G E} of the /i-trace process if h 
belongs to L^{fJ.). Indeed, by (|6.12p and 



= 5:Af(r7)P4T+=T+^^^j] < 00. 

Therefore, assumption (|6.14p holds for the /i-trace process whenever h belongs to 
L^{^). In this case, its capacity, denoted by cap;j(', •), is well defined. Next result 
shows a simple relation between cap/j(', •) and the capacity of the original process. 

Lemma 6.9. Let h : E ^ R+ be a nonnegative ii-integrable function with nonemp- 
ty support denoted by F. Then, for every subsets A, B of F, An B = 0, 

(h) f_, ca.p,^{A, B) = cap{A,B). 

Proof. Fix a function h : E K+ with the properties required in the statement of 
the lemma and two subsets A, B oi F such that AnB ^ 0. By Lemma 15^ applied 
to the process {r]^ : t > 0} and by identities (EH), ((6TT2)l and ((O)) . 

capJAB) ^ ^A^\r;)A'Xry)P4r+(r;'')<r+(r7'')] 

Since for 77 G A, the event {Tg < T^} is contained in the event {Tp — ?F\{r;}} 
P^-almost surely, the previous expression is equal to 

-^5:M(r;)P4T+<Tl]. 

By Lemma l6.6l this expression is equal to {h)~^ca.p{A, B), which proves the lemma. 

□ 

We conclude this subsection proving a relation between expectations of time 
integrals of functions and capacities. Fix two disjoint subsets A, B of E. Define 
the probability measure vab on A as 

M(77)P4r+<r+] 

vab{v) = TT~^\ £ ^• 

cap(yi, B) 

Denote by Ei/^^ the expectation associated to the Markov process {qt : i > 0} with 
initial distribution vab ■ The proof of the following proposition is an adaptation of 
the proof of identity (4.28) in [12]. 

Proposition 6.10. Fix two disjoint subsets A, B of E. Let g : E ^ M. be a 

fi-integrable function. Then, 



9iVt)dt = 
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Proof. Wc first claim that the proposition holds for indicator functions of states. 
Fix an arbitrary state £, G E. Consider the random time ts ■= infjrt > : Yn € B} 
and the last exit time 

Lab ■= sup{7i > E A and n < ts} 

with the convention that sup0 = — oo. Then, 

P^[Ta<Tb] = Y.^dLAB=n] 

n>0 

n>0 TjeA 

r]£A n>0 

Since Y is reversible with respect to M, the last expression is equal to 

Recall from the beginning of this section that {e„ : ti > 0} is a sequence of i.i.d. 
mean one exponential random variables independent of the jump chain {Yn : n > 0}. 
By definition of the measure vab, this sum can be rewritten as 

cap(A, B) ^ .Mv) ^ E, [ ^ l{r„ = a " 



cap{A, B) 



This proves the assertion for g — 1{^}. By linearity and the monotone convergence 
theorem we get the desired result for positive and then /i-integrable functions. □ 



In particular, taking A = {rj} and B = {^} for rj ^ ^ we have that 



E, 



g{ris)ds 



cM{v},m 



(6.16) 



for any /x-integrable function g. 

This formula provides a more accurate estimate in Corollary 16. 51 in the reversible 
context. For each x E S, let 

cap(^^) := inf cap({77}, {^^^1) ■ 

Lemma 6.11. Let g : E ^ R be a function integrable with respect to fi. If the 
measure fj, is reversible then, for each x E S, 



sup 



< 



cap(^2; 



■ ^,{An 



13^(77,)! ds 
where \g\{ri) = \g{ri)\ for all rj in E. 

Proof. By (|6.16[) and the fact that < f{ri}{^-,} < Ij the left hand side is bounded 
above by 



veA"\u-} cap({?7},{Cx}) 



sup 



< 



(|g^l)M 



< 



2{\g\l{A-}), 
capi^x) 
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for each a; G 5. This completes the proof. □ 
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